Error running ds002168 (example) subject: Potentially with symbolic links?

Question

Error running ds002168 (example) subject: Potentially with symbolic links?

jlhanson5 opened this issue 9 months ago · comments

Hello HippUnfold colleagues,

I ran into issues just test driving the singularity version of the program. I'm on a linux server (Linux 5.4.0-150-generic x86_64) and the singularity container starts but then crashes shortly after that.

After downloading example data, I start things with this command:
singularity run -e khanlab_hippunfold_latest.sif ds002168 ds002168_hippunfold participant -p --cores 8 --modality T1w
(which is fairly standard to my understanding, just changing the number of cores)

I have attached the full command line output here, but I think this is where errors start popping up...

Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Select jobs to execute...
wb_command -volume-to-surface-mapping work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii temp.shape.gii -enclosing  && wb_command -metric-label-import temp.shape.gii /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii && wb_command -set-structure hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii CORTEX_LEFT
    /home/jamielh/.cache/hippunfold/trained_model.3d_full   0%[                                                                                                                                      ]   3.15M   629KB/s    eta 64m 34sTraceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/jobs.py", line 931, in prepare
    os.symlink(original, link)
OSError: [Errno 95] Operation not supported: '../../../../../../work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz' -> '/home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/shadow/tmpm4evs4g4/work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz'
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Select jobs to execute...
wb_command -volume-to-surface-mapping work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii temp.shape.gii -enclosing  && wb_command -metric-label-import temp.shape.gii /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii && wb_command -set-structure hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii CORTEX_RIGHT
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/jobs.py", line 931, in prepare
    os.symlink(original, link)
OSError: [Errno 95] Operation not supported: '../../../../../../work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz' -> '/home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/shadow/tmpujecq86s/work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz'
che/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T  13%[=================>                                                                                                                    ] 326.58M  3.85MB/s    eta 8m 13s [Thu Oct  5 11:24:20 2023]
Finished job 27.
41 of 251 steps (16%) done
/home/jamielh/.cache/hippunfold/trained_model.3d_fullres. 100%[=====================================================================================================================================>]   2.28G  4.06MB/s    in 10m 20s

2023-10-05 11:33:20 (3.77 MB/s) - ‘/home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar’ saved [2450872320/2450872320]

[Thu Oct  5 11:33:20 2023]
Finished job 25.
42 of 251 steps (17%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-10-05T112252.716645.snakemake.log

I think that's an error in making symbolic links (which isn't allowed on our server... as our app server is linux, but our file server is windows... so symbolic links create issues across platforms, or so our IT says).

Is that actually the issue? Have others run into this program? Our team could easily edit the code to make hard copies of lots of file (space isn't necessarily a huge constraint). BUT I thought I would post and get people's thoughts or advice.

Any ideas are much appreciated!
Jamie.

full_cl_output.txt

Jordan DeKraker · Answer 1 · Mon Oct 16 2023 22:17:41 GMT+0800 (China Standard Time)

Hmm I've never seen this error before. Sorry for the inconvenience, but would you mind running the same thing again with the --keep_going flag included? Attaching the log file again is also appreciated!

HippUnfold commands don't directly make any symlinks (unless I've missed something somewhere), but it might be happening under the hood in one of the wb_command calls.

jlhanson5 · Answer 2 · Mon Oct 16 2023 22:33:54 GMT+0800 (China Standard Time)

Not a problem and thanks for following up.

I ran:
singularity run -e khanlab_hippunfold_latest.sif ds002168 ds002168_hippunfold participant -p --cores 8 --modality T1w --keep-going

And here's the FULL output I got:

Config file config/snakebids.yml is extended by additional config specified via the command line.
Wildcard constraints in inputs are ignored. (rule: template_xfm_itk2ras)
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Job stats:
job                                       count
--------------------------------------  -------
affine_gii_to_native                          8
all                                           1
archive_work_after_final                      1
calculate_curvature_from_surface1             2
calculate_curvature_from_surface2             2
calculate_gyrification1                       2
calculate_gyrification2                       4
calculate_surface_area1                       2
calculate_surface_area2                       4
calculate_thickness_from_surface1             2
calculate_thickness_from_surface2             2
combine_tissue_subfield_labels_corobl         2
compose_template_xfm_corobl                   1
compose_warps_native_to_unfold                4
compose_warps_unfold_to_crop_native           4
convert_template_xfm_ras2itk                  1
correct_nan_vertices1                         6
correct_nan_vertices2                         8
cp_unfolded_noconstrain                       2
create_dlabel_cifti_subfields                 1
create_dscalar_metric_cifti                   5
create_native_coord_ref                       4
create_native_crop_ref                        2
create_spec_file_dentate                      2
create_spec_file_hipp                         2
create_warps_dentate                          2
create_warps_hipp                             2
equivolume_coords                             2
expand_unfolded_warps                         4
get_subfield_vols_subj                        1
inject_init_laplace_coords                   12
label_subfields_from_vol_coords_corobl        2
laplace_coords_dentate                        6
laplace_coords_hipp                           6
lr_flip_t1                                    1
merge_lr_spec_file                            2
metric_to_nii                                 6
n4_t1                                         1
nii_to_label_gii                              2
normalize_curvature1                          2
normalize_curvature2                          4
plot_subj_subfields                           1
prep_equivolume_coords                        2
prep_segs_for_greedy                          2
qc_nnunet_dice                                2
qc_nnunet_f3d                                 2
qc_reg_to_template                            1
qc_subfield                                   2
qc_subfield_surf                              4
reg_to_template                               1
reinsert_subject_labels                       2
resample_coords_native_crop                  14
resample_native_to_crop                       2
resample_subfields_native_crop                2
resample_subfields_to_native                  2
resample_unfoldreg_subfields                  2
run_inference                                 2
template_shape_inject                         2
template_shape_reg                            2
template_xfm_itk2ras                          1
unflip_init_coords                            6
unflip_nnunet_nii                             1
unflip_postproc                               1
unfolded_registration                         2
warp_gii_unfold2corobl1                       6
warp_gii_unfold2corobl2                       8
warp_gii_unfoldreg                            6
warp_t1_to_corobl_crop                        2
total                                       209

Select jobs to execute...

[Mon Oct 16 10:25:04 2023]
rule n4_t1:
    input: work/sub-1425/anat/sub-1425_T1w.nii.gz
    output: hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz
    jobid: 20
    reason: Missing output files: hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz
    wildcards: subject=1425
    threads: 8
    resources: tmpdir=/tmp

ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=8 N4BiasFieldCorrection -d 3 -i work/sub-1425/anat/sub-1425_T1w.nii.gz -o hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz
[Mon Oct 16 10:26:03 2023]
Finished job 20.
1 of 209 steps (0.5%) done
Select jobs to execute...

[Mon Oct 16 10:26:03 2023]
rule reg_to_template:
    input: hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/T1w_head_700um.nii.gz, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/identity_xfm.txt
    output: work/sub-1425/anat/sub-1425_space-CITI168_desc-affine_T1w.nii.gz, work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-ras_xfm.txt
    jobid: 24
    reason: Missing output files: work/sub-1425/anat/sub-1425_space-CITI168_desc-affine_T1w.nii.gz, work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-ras_xfm.txt; Input files updated by another job: hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz
    wildcards: subject=1425
    resources: tmpdir=/tmp

reg_aladin -flo hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz -ref /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/T1w_head_700um.nii.gz -res work/sub-1425/anat/sub-1425_space-CITI168_desc-affine_T1w.nii.gz -aff work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-ras_xfm.txt

[Mon Oct 16 10:26:03 2023]
rule nii_to_label_gii:
    input: work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt
    output: hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii
    jobid: 78
    reason: Missing output files: hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii
    wildcards: subject=1425, hemi=L, space=T1w, density=0p5mm, atlas=multihist7
    resources: tmpdir=/tmp

wb_command -volume-to-surface-mapping work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii temp.shape.gii -enclosing  && wb_command -metric-label-import temp.shape.gii /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii && wb_command -set-structure hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii CORTEX_LEFT

[Mon Oct 16 10:26:03 2023]
rule nii_to_label_gii:
    input: work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt
    output: hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii
    jobid: 143
    reason: Missing output files: hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii
    wildcards: subject=1425, hemi=R, space=T1w, density=0p5mm, atlas=multihist7
    resources: tmpdir=/tmp

wb_command -volume-to-surface-mapping work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii temp.shape.gii -enclosing  && wb_command -metric-label-import temp.shape.gii /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii && wb_command -set-structure hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii CORTEX_RIGHT

[NiftyReg ALADIN] Command line:
	 reg_aladin -flo hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz -ref /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/T1w_head_700um.nii.gz -res work/sub-1425/anat/sub-1425_space-CITI168_desc-affine_T1w.nii.gz -aff work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-ras_xfm.txt

Config file config/snakebids.yml is extended by additional config specified via the command line.
Config file config/snakebids.yml is extended by additional config specified via the command line.
[reg_aladin] Parameters
[reg_aladin] Reference image name: /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/T1w_head_700um.nii.gz
[reg_aladin] 	198x263x212 voxels
[reg_aladin] 	0.7x0.7x0.7 mm
[reg_aladin] floating image name: hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz
[reg_aladin] 	192x256x256 voxels
[reg_aladin] 	1x1x1 mm
[reg_aladin] Maximum iteration number: 5 (10 during the first level)
[reg_aladin] Percentage of blocks: 50 % (100% during the first level)
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Select jobs to execute...
wb_command -volume-to-surface-mapping work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii temp.shape.gii -enclosing  && wb_command -metric-label-import temp.shape.gii /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii && wb_command -set-structure hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii CORTEX_RIGHT
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Select jobs to execute...
wb_command -volume-to-surface-mapping work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii temp.shape.gii -enclosing  && wb_command -metric-label-import temp.shape.gii /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii && wb_command -set-structure hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii CORTEX_LEFT
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/jobs.py", line 931, in prepare
    os.symlink(original, link)
OSError: [Errno 95] Operation not supported: '../../../../../../work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz' -> '/home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/shadow/tmphoq8y7sg/work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz'
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/jobs.py", line 931, in prepare
    os.symlink(original, link)
OSError: [Errno 95] Operation not supported: '../../../../../../work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz' -> '/home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/shadow/tmpo16ob6n4/work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz'
[reg_aladin] Current level 1 / 3
[reg_aladin] reference image size: 	49x65x53 voxels	2.8x2.8x2.8 mm
[reg_aladin] floating image size: 	48x64x64 voxels	4.00002x4x4 mm
[reg_aladin] Block size = [4 4 4]
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
[reg_aladin] Block number = [13 17 14]
[reg_aladin] Initial transformation matrix::
1	0	0	-4.33524
0	1	0	67.023
0	0	1	-1.79366
0	0	0	1
[reg_aladin] Final transformation matrix::
0.990171	0.00817884	0.0203503	-4.93068
-0.00755474	0.991661	-0.128698	43.1019
-0.0246277	0.129717	0.954492	10.233
0	0	0	1
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[reg_aladin] Current level 2 / 3
[reg_aladin] reference image size: 	99x131x106 voxels	1.4x1.4x1.4 mm
[reg_aladin] floating image size: 	96x128x128 voxels	2.00001x2x2 mm
[reg_aladin] Block size = [4 4 4]
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
[reg_aladin] Block number = [25 33 27]
[reg_aladin] Initial transformation matrix::
0.990171	0.00817884	0.0203503	-4.93068
-0.00755474	0.991661	-0.128698	43.1019
-0.0246277	0.129717	0.954492	10.233
0	0	0	1
[reg_aladin] Final transformation matrix::
0.987908	0.0105682	0.0068881	-5.17942
-0.0133402	0.990277	-0.140081	43.4797
-0.023193	0.143957	0.933565	10.7379
0	0	0	1
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[reg_aladin] Current level 3 / 3
[reg_aladin] reference image size: 	198x263x212 voxels	0.7x0.7x0.7 mm
[reg_aladin] floating image size: 	192x256x256 voxels	1x1x1 mm
[reg_aladin] Block size = [4 4 4]
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
[reg_aladin] Block number = [50 66 53]
[reg_aladin] Initial transformation matrix::
0.987908	0.0105682	0.0068881	-5.17942
-0.0133402	0.990277	-0.140081	43.4797
-0.023193	0.143957	0.933565	10.7379
0	0	0	1
[reg_aladin] Final transformation matrix::
0.986664	0.0136748	0.014529	-5.17517
-0.00926125	1.00028	-0.153429	43.351
-0.0215617	0.142466	0.945737	10.8306
0	0	0	1
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Registration Performed in 3 min 2 sec
Have a good day !
[Mon Oct 16 10:29:05 2023]
Finished job 24.
2 of 209 steps (1%) done
Select jobs to execute...

[Mon Oct 16 10:29:05 2023]
rule convert_template_xfm_ras2itk:
    input: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-ras_xfm.txt
    output: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-itk_xfm.txt
    jobid: 23
    reason: Missing output files: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-itk_xfm.txt; Input files updated by another job: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-ras_xfm.txt
    wildcards: subject=1425, reg_suffix=T1w
    resources: tmpdir=/tmp

c3d_affine_tool work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-ras_xfm.txt  -oitk work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-itk_xfm.txt

[Mon Oct 16 10:29:05 2023]
rule qc_reg_to_template:
    input: /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/T1w_head_700um.nii.gz, work/sub-1425/anat/sub-1425_space-CITI168_desc-affine_T1w.nii.gz
    output: hippunfold/sub-1425/qc/sub-1425_from-T1w_to-CITI168_regqc.png
    jobid: 238
    reason: Missing output files: hippunfold/sub-1425/qc/sub-1425_from-T1w_to-CITI168_regqc.png; Input files updated by another job: work/sub-1425/anat/sub-1425_space-CITI168_desc-affine_T1w.nii.gz
    wildcards: subject=1425, native_modality=T1w
    resources: tmpdir=/tmp

/opt/conda/bin/python /home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/scripts/tmp1usdqf4s.vis_regqc.py
[Mon Oct 16 10:29:07 2023]
Finished job 23.
3 of 209 steps (1%) done
Select jobs to execute...

[Mon Oct 16 10:29:07 2023]
rule compose_template_xfm_corobl:
    input: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-itk_xfm.txt, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/CoronalOblique_rigid.txt
    output: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt
    jobid: 22
    reason: Missing output files: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt; Input files updated by another job: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-itk_xfm.txt
    wildcards: subject=1425
    resources: tmpdir=/tmp

c3d_affine_tool -itk work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-CITI168_type-itk_xfm.txt -itk /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/CoronalOblique_rigid.txt -mult -oitk work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt
[Mon Oct 16 10:29:07 2023]
Finished job 22.
4 of 209 steps (2%) done
Select jobs to execute...

[Mon Oct 16 10:29:07 2023]
rule warp_t1_to_corobl_crop:
    input: hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz, work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/T2w_300umCoronalOblique_hemi-R.nii.gz, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/CoronalOblique_rigid.txt
    output: work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz
    jobid: 92
    reason: Missing output files: work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz; Input files updated by another job: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt, hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz
    wildcards: subject=1425, hemi=R
    resources: tmpdir=/tmp

ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=1 antsApplyTransforms -d 3 --interpolation Linear -i hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz -o work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz -r /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/T2w_300umCoronalOblique_hemi-R.nii.gz  -t work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt

[Mon Oct 16 10:29:07 2023]
rule warp_t1_to_corobl_crop:
    input: hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz, work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/T2w_300umCoronalOblique_hemi-L.nii.gz, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/CoronalOblique_rigid.txt
    output: work/sub-1425/anat/sub-1425_hemi-L_space-corobl_desc-preproc_T1w.nii.gz
    jobid: 19
    reason: Missing output files: work/sub-1425/anat/sub-1425_hemi-L_space-corobl_desc-preproc_T1w.nii.gz; Input files updated by another job: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt, hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz
    wildcards: subject=1425, hemi=L
    resources: tmpdir=/tmp

ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=1 antsApplyTransforms -d 3 --interpolation Linear -i hippunfold/sub-1425/anat/sub-1425_desc-preproc_T1w.nii.gz -o work/sub-1425/anat/sub-1425_hemi-L_space-corobl_desc-preproc_T1w.nii.gz -r /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/CITI168/T2w_300umCoronalOblique_hemi-L.nii.gz  -t work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt

[Mon Oct 16 10:29:07 2023]
rule template_xfm_itk2ras:
    input: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt
    output: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-ras_xfm.txt
    jobid: 62
    reason: Missing output files: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-ras_xfm.txt; Input files updated by another job: work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt
    wildcards: subject=1425, native_modality=T1w
    resources: tmpdir=/tmp

c3d_affine_tool -itk work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-itk_xfm.txt -o work/sub-1425/warps/sub-1425_desc-affine_from-T1w_to-corobl_type-ras_xfm.txt
[Mon Oct 16 10:29:07 2023]
Finished job 62.
5 of 209 steps (2%) done
[Mon Oct 16 10:29:14 2023]
Finished job 92.
6 of 209 steps (3%) done
Select jobs to execute...
[Mon Oct 16 10:29:14 2023]
Finished job 19.
7 of 209 steps (3%) done

[Mon Oct 16 10:29:14 2023]
rule lr_flip_t1:
    input: work/sub-1425/anat/sub-1425_hemi-L_space-corobl_desc-preproc_T1w.nii.gz
    output: work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz
    jobid: 18
    reason: Missing output files: work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz; Input files updated by another job: work/sub-1425/anat/sub-1425_hemi-L_space-corobl_desc-preproc_T1w.nii.gz
    wildcards: subject=1425, hemi=L, desc=preproc
    resources: tmpdir=/tmp

c3d work/sub-1425/anat/sub-1425_hemi-L_space-corobl_desc-preproc_T1w.nii.gz -flip x -o  work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz
/opt/conda/lib/python3.9/site-packages/nilearn/datasets/__init__.py:93: FutureWarning: Fetchers from the nilearn.datasets module will be updated in version 0.9 to return python strings instead of bytes and Pandas dataframes instead of Numpy arrays.
  warn("Fetchers from the nilearn.datasets module will be "
[Mon Oct 16 10:29:18 2023]
Finished job 18.
8 of 209 steps (4%) done
Select jobs to execute...
[Mon Oct 16 10:29:26 2023]
Finished job 238.
9 of 209 steps (4%) done

[Mon Oct 16 10:29:26 2023]
rule run_inference:
    input: work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz, /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar
    output: work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-nnunet_dseg.nii.gz
    log: logs/sub-1425/sub-1425_hemi-R_space-corobl_nnunet.txt
    jobid: 91
    reason: Missing output files: work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-nnunet_dseg.nii.gz; Input files updated by another job: work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz
    wildcards: subject=1425, hemi=R
    threads: 8
    resources: tmpdir=/tmp, gpus=0, mem_mb=16000, mem_mib=15259, time=60

mkdir -p tempmodel tempimg templbl && cp work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz tempimg/temp_0000.nii.gz && tar -xf /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar -C tempmodel && export RESULTS_FOLDER=tempmodel && export nnUNet_n_proc_DA=8 && nnUNet_predict -i tempimg -o templbl -t Task101_hcp1200_T1w -chk model_best -tr nnUNetTrainerV2 --disable_tta &> logs/sub-1425/sub-1425_hemi-R_space-corobl_nnunet.txt && cp templbl/temp.nii.gz work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-nnunet_dseg.nii.gz
Config file config/snakebids.yml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Provided resources: gpus=0, mem_mb=16000, mem_mib=15259, time=60
Singularity containers: ignored
Select jobs to execute...
mkdir -p tempmodel tempimg templbl && cp work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz tempimg/temp_0000.nii.gz && tar -xf /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar -C tempmodel && export RESULTS_FOLDER=tempmodel && export nnUNet_n_proc_DA=8 && nnUNet_predict -i tempimg -o templbl -t Task101_hcp1200_T1w -chk model_best -tr nnUNetTrainerV2 --disable_tta &> logs/sub-1425/sub-1425_hemi-R_space-corobl_nnunet.txt && cp templbl/temp.nii.gz work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-nnunet_dseg.nii.gz
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/jobs.py", line 931, in prepare
    os.symlink(original, link)
OSError: [Errno 95] Operation not supported: '../../../../../../work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz' -> '/home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/shadow/tmpstz7l964/work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz'
Select jobs to execute...

[Mon Oct 16 10:29:28 2023]
rule run_inference:
    input: work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz, /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar
    output: work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-nnunet_dseg.nii.gz
    log: logs/sub-1425/sub-1425_hemi-Lflip_space-corobl_nnunet.txt
    jobid: 17
    reason: Missing output files: work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-nnunet_dseg.nii.gz; Input files updated by another job: work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz
    wildcards: subject=1425, hemi=Lflip
    threads: 8
    resources: tmpdir=/tmp, gpus=0, mem_mb=16000, mem_mib=15259, time=60

mkdir -p tempmodel tempimg templbl && cp work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz tempimg/temp_0000.nii.gz && tar -xf /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar -C tempmodel && export RESULTS_FOLDER=tempmodel && export nnUNet_n_proc_DA=8 && nnUNet_predict -i tempimg -o templbl -t Task101_hcp1200_T1w -chk model_best -tr nnUNetTrainerV2 --disable_tta &> logs/sub-1425/sub-1425_hemi-Lflip_space-corobl_nnunet.txt && cp templbl/temp.nii.gz work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-nnunet_dseg.nii.gz
Config file config/snakebids.yml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Provided resources: gpus=0, mem_mb=16000, mem_mib=15259, time=60
Singularity containers: ignored
Select jobs to execute...
mkdir -p tempmodel tempimg templbl && cp work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz tempimg/temp_0000.nii.gz && tar -xf /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar -C tempmodel && export RESULTS_FOLDER=tempmodel && export nnUNet_n_proc_DA=8 && nnUNet_predict -i tempimg -o templbl -t Task101_hcp1200_T1w -chk model_best -tr nnUNetTrainerV2 --disable_tta &> logs/sub-1425/sub-1425_hemi-Lflip_space-corobl_nnunet.txt && cp templbl/temp.nii.gz work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-nnunet_dseg.nii.gz
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/jobs.py", line 931, in prepare
    os.symlink(original, link)
OSError: [Errno 95] Operation not supported: '../../../../../../work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz' -> '/home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/shadow/tmpf5xac4t2/work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz'
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-10-16T102459.125995.snakemake.log

Thoughts on all that?

Jordan DeKraker · Answer 3 · Mon Oct 16 2023 22:49:02 GMT+0800 (China Standard Time)

hmm it may be the case that the hippunfold container can read and write into the relative path ../../../../../ds002168_hippunfold/work/sub-1425, but not the absolute path /home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold

could you try again after running
export SINGULARITY_BINDPATH=/home:/home
please?

jlhanson5 · Answer 4 · Mon Oct 16 2023 22:53:14 GMT+0800 (China Standard Time)

Thanks for this help and assistance!

Just as a quick clarification / note of background: We have app and files servers, and the file server is mounted (when I log into the app server). BUT I can get in trouble if I write to my app server (as space is limited). So just to clarify, I presuming that export command won't write to my actual home directory (just singularity/docker's home)?

Jordan DeKraker · Answer 5 · Mon Oct 16 2023 23:00:38 GMT+0800 (China Standard Time)

That would add your home directory. If you are running this singularity command from your home directory, then the output folder ds002168_hippunfold will also be in your home directory. I would recommend doing this instead:

singularity run -e khanlab_hippunfold_latest.sif ds002168 <yourserverdirectory>/ds002168_hippunfold participant -p --cores 8 --modality T1w --keep-going

in which case you could first run

export SINGULARITY_BINDPATH=<yourserverdirectory>:<yourserverdirectory>

make sure to fill in <yourserverdirectory>

jlhanson5 · Answer 6 · Mon Oct 16 2023 23:03:51 GMT+0800 (China Standard Time)

No luck yet...

I ran this first... and that looks fine (and I can read/write there).

(base) jamielh@pfc:~/Volumes/Hanson/Resources$ export SINGULARITY_BINDPATH=/home/jamielh/Volumes/Hanson/Resources/:/home/jamielh/Volumes/Hanson/Resources/
(base) jamielh@pfc:~/Volumes/Hanson/Resources$ echo $SINGULARITY_BINDPATH
/home/jamielh/Volumes/Hanson/Resources/:/home/jamielh/Volumes/Hanson/Resources/

But the error/output I get is:

(base) jamielh@pfc:~/Volumes/Hanson/Resources$ singularity run -e khanlab_hippunfold_latest.sif ds002168 ds002168_hippunfold participant -p --cores 8 --modality T1w --keep-going
Config file config/snakebids.yml is extended by additional config specified via the command line.
Wildcard constraints in inputs are ignored. (rule: template_xfm_itk2ras)
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Job stats:
job                                       count
--------------------------------------  -------
affine_gii_to_native                          8
all                                           1
archive_work_after_final                      1
calculate_curvature_from_surface1             2
calculate_curvature_from_surface2             2
calculate_gyrification1                       2
calculate_gyrification2                       4
calculate_surface_area1                       2
calculate_surface_area2                       4
calculate_thickness_from_surface1             2
calculate_thickness_from_surface2             2
combine_tissue_subfield_labels_corobl         2
compose_warps_native_to_unfold                4
compose_warps_unfold_to_crop_native           4
correct_nan_vertices1                         6
correct_nan_vertices2                         8
cp_unfolded_noconstrain                       2
create_dlabel_cifti_subfields                 1
create_dscalar_metric_cifti                   5
create_native_coord_ref                       4
create_native_crop_ref                        2
create_spec_file_dentate                      2
create_spec_file_hipp                         2
create_warps_dentate                          2
create_warps_hipp                             2
equivolume_coords                             2
expand_unfolded_warps                         4
get_subfield_vols_subj                        1
inject_init_laplace_coords                   12
label_subfields_from_vol_coords_corobl        2
laplace_coords_dentate                        6
laplace_coords_hipp                           6
merge_lr_spec_file                            2
metric_to_nii                                 6
nii_to_label_gii                              2
normalize_curvature1                          2
normalize_curvature2                          4
plot_subj_subfields                           1
prep_equivolume_coords                        2
prep_segs_for_greedy                          2
qc_nnunet_dice                                2
qc_nnunet_f3d                                 2
qc_subfield                                   2
qc_subfield_surf                              4
reinsert_subject_labels                       2
resample_coords_native_crop                  14
resample_native_to_crop                       2
resample_subfields_native_crop                2
resample_subfields_to_native                  2
resample_unfoldreg_subfields                  2
run_inference                                 2
template_shape_inject                         2
template_shape_reg                            2
unflip_init_coords                            6
unflip_nnunet_nii                             1
unflip_postproc                               1
unfolded_registration                         2
warp_gii_unfold2corobl1                       6
warp_gii_unfold2corobl2                       8
warp_gii_unfoldreg                            6
total                                       200

Select jobs to execute...

[Mon Oct 16 11:02:08 2023]
rule run_inference:
    input: work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz, /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar
    output: work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-nnunet_dseg.nii.gz
    log: logs/sub-1425/sub-1425_hemi-R_space-corobl_nnunet.txt
    jobid: 91
    reason: Missing output files: work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-nnunet_dseg.nii.gz
    wildcards: subject=1425, hemi=R
    threads: 8
    resources: tmpdir=/tmp, gpus=0, mem_mb=16000, mem_mib=15259, time=60

mkdir -p tempmodel tempimg templbl && cp work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz tempimg/temp_0000.nii.gz && tar -xf /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar -C tempmodel && export RESULTS_FOLDER=tempmodel && export nnUNet_n_proc_DA=8 && nnUNet_predict -i tempimg -o templbl -t Task101_hcp1200_T1w -chk model_best -tr nnUNetTrainerV2 --disable_tta &> logs/sub-1425/sub-1425_hemi-R_space-corobl_nnunet.txt && cp templbl/temp.nii.gz work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-nnunet_dseg.nii.gz
Config file config/snakebids.yml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Provided resources: gpus=0, mem_mb=16000, mem_mib=15259, time=60
Singularity containers: ignored
Select jobs to execute...
mkdir -p tempmodel tempimg templbl && cp work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz tempimg/temp_0000.nii.gz && tar -xf /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar -C tempmodel && export RESULTS_FOLDER=tempmodel && export nnUNet_n_proc_DA=8 && nnUNet_predict -i tempimg -o templbl -t Task101_hcp1200_T1w -chk model_best -tr nnUNetTrainerV2 --disable_tta &> logs/sub-1425/sub-1425_hemi-R_space-corobl_nnunet.txt && cp templbl/temp.nii.gz work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-nnunet_dseg.nii.gz
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/jobs.py", line 931, in prepare
    os.symlink(original, link)
OSError: [Errno 95] Operation not supported: '../../../../../../work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz' -> '/home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/shadow/tmptyvhfsod/work/sub-1425/anat/sub-1425_hemi-R_space-corobl_desc-preproc_T1w.nii.gz'
Select jobs to execute...

[Mon Oct 16 11:02:10 2023]
rule run_inference:
    input: work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz, /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar
    output: work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-nnunet_dseg.nii.gz
    log: logs/sub-1425/sub-1425_hemi-Lflip_space-corobl_nnunet.txt
    jobid: 17
    reason: Missing output files: work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-nnunet_dseg.nii.gz
    wildcards: subject=1425, hemi=Lflip
    threads: 8
    resources: tmpdir=/tmp, gpus=0, mem_mb=16000, mem_mib=15259, time=60

mkdir -p tempmodel tempimg templbl && cp work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz tempimg/temp_0000.nii.gz && tar -xf /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar -C tempmodel && export RESULTS_FOLDER=tempmodel && export nnUNet_n_proc_DA=8 && nnUNet_predict -i tempimg -o templbl -t Task101_hcp1200_T1w -chk model_best -tr nnUNetTrainerV2 --disable_tta &> logs/sub-1425/sub-1425_hemi-Lflip_space-corobl_nnunet.txt && cp templbl/temp.nii.gz work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-nnunet_dseg.nii.gz
Config file config/snakebids.yml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Provided resources: gpus=0, mem_mb=16000, mem_mib=15259, time=60
Singularity containers: ignored
Select jobs to execute...
mkdir -p tempmodel tempimg templbl && cp work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz tempimg/temp_0000.nii.gz && tar -xf /home/jamielh/.cache/hippunfold/trained_model.3d_fullres.Task101_hcp1200_T1w.nnUNetTrainerV2.model_best.tar -C tempmodel && export RESULTS_FOLDER=tempmodel && export nnUNet_n_proc_DA=8 && nnUNet_predict -i tempimg -o templbl -t Task101_hcp1200_T1w -chk model_best -tr nnUNetTrainerV2 --disable_tta &> logs/sub-1425/sub-1425_hemi-Lflip_space-corobl_nnunet.txt && cp templbl/temp.nii.gz work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-nnunet_dseg.nii.gz
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/jobs.py", line 931, in prepare
    os.symlink(original, link)
OSError: [Errno 95] Operation not supported: '../../../../../../work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz' -> '/home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/shadow/tmpu85hfmzr/work/sub-1425/anat/sub-1425_hemi-Lflip_space-corobl_desc-preproc_T1w.nii.gz'
Select jobs to execute...

[Mon Oct 16 11:02:12 2023]
rule nii_to_label_gii:
    input: work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt
    output: hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii
    jobid: 78
    reason: Missing output files: hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii
    wildcards: subject=1425, hemi=L, space=T1w, density=0p5mm, atlas=multihist7
    resources: tmpdir=/tmp

wb_command -volume-to-surface-mapping work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii temp.shape.gii -enclosing  && wb_command -metric-label-import temp.shape.gii /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii && wb_command -set-structure hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii CORTEX_LEFT

[Mon Oct 16 11:02:12 2023]
rule nii_to_label_gii:
    input: work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii, /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt
    output: hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii
    jobid: 143
    reason: Missing output files: hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii
    wildcards: subject=1425, hemi=R, space=T1w, density=0p5mm, atlas=multihist7
    resources: tmpdir=/tmp

wb_command -volume-to-surface-mapping work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii temp.shape.gii -enclosing  && wb_command -metric-label-import temp.shape.gii /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii && wb_command -set-structure hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii CORTEX_RIGHT
Config file config/snakebids.yml is extended by additional config specified via the command line.
Config file config/snakebids.yml is extended by additional config specified via the command line.
Building DAG of jobs...
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Select jobs to execute...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Select jobs to execute...
wb_command -volume-to-surface-mapping work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii temp.shape.gii -enclosing  && wb_command -metric-label-import temp.shape.gii /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii && wb_command -set-structure hippunfold/sub-1425/surf/sub-1425_hemi-L_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii CORTEX_LEFT
wb_command -volume-to-surface-mapping work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/unfold_template_hipp/tpl-avg_space-unfold_den-0p5mm_midthickness.surf.gii temp.shape.gii -enclosing  && wb_command -metric-label-import temp.shape.gii /opt/conda/lib/python3.9/site-packages/hippunfold/workflow/../resources/multihist7/labellist.txt hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii && wb_command -set-structure hippunfold/sub-1425/surf/sub-1425_hemi-R_space-T1w_den-0p5mm_label-hipp_atlas-multihist7_subfields.label.gii CORTEX_RIGHT
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/jobs.py", line 931, in prepare
    os.symlink(original, link)
OSError: [Errno 95] Operation not supported: '../../../../../../work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz' -> '/home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/shadow/tmp8epkxm22/work/sub-1425/anat/sub-1425_hemi-L_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz'
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/opt/conda/lib/python3.9/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "/opt/conda/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "/opt/conda/lib/python3.9/site-packages/snakemake/jobs.py", line 931, in prepare
    os.symlink(original, link)
OSError: [Errno 95] Operation not supported: '../../../../../../work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz' -> '/home/jamielh/Volumes/Hanson/Resources/ds002168_hippunfold/.snakemake/shadow/tmp0zvso31x/work/sub-1425/anat/sub-1425_hemi-R_space-unfold_label-hipp_atlas-multihist7_subfields.nii.gz'
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-10-16T110204.795636.snakemake.log

Other thoughts?

Jordan DeKraker · Answer 7 · Mon Oct 16 2023 23:07:38 GMT+0800 (China Standard Time)

what do you get if you run pwd -P?

jlhanson5 · Answer 8 · Mon Oct 16 2023 23:09:19 GMT+0800 (China Standard Time)

/home/jamielh/Volumes/Hanson/Resources
?

Jordan DeKraker · Answer 9 · Mon Oct 16 2023 23:20:20 GMT+0800 (China Standard Time)

Hmmm OK these mounted volume paths and permissions can be tricky. Let me break for lunch and then I'll try to find you a solution

Jordan DeKraker · Answer 10 · Mon Oct 16 2023 23:34:30 GMT+0800 (China Standard Time)

Ahh OK I found the problem, and its not with paths or mounts. Its that snakemake creates symlinks for temporary files in "shadow" directories. Since symlinks are banned, the snakemake rule is failing.

I don't currently see a way to disable "shadow" for snakemake. I don't really want to remove the shadow feature in hippunfold, since its really useful for cleaning up temporary files.

I'll think/read about it and try to find a solution soon

Ali Khan · Answer 11 · Mon Oct 16 2023 23:34:36 GMT+0800 (China Standard Time)

Hi guys, Unfortunately hippunfold as-is requires symlinks, as snakemake uses symlinks when a rule is run with the shadow option. There is a workaround if you want to avoid this, need to replace "minimal" with "copy-minimal" in all the shadow directives in the .smk files. I think a simple find replace would work fine. Fyi to run a modified hippunfold easily you can just use singularity exec instead of singularity run, and point it to `hippunfold/run.py` in the modified git repository, and use the same container you have. Best, Ali

…

________________________________ From: Jordan DeKraker ***@***.***> Sent: Monday, October 16, 2023 11:20:30 AM To: khanlab/hippunfold ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [khanlab/hippunfold] Error running ds002168 (example) subject: Potentially with symbolic links? (Issue #266) Hmmm OK these mounted volume paths and permissions can be tricky. Let me break for lunch and then I'll try to find you a solution — Reply to this email directly, view it on GitHub<#266 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACXV2XLRE7UDLSAWSMBKVSTX7VGD5AVCNFSM6AAAAAA5USNKL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRUG4ZDMNJWGY>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

Jordan DeKraker · Answer 12 · Mon Oct 16 2023 23:37:04 GMT+0800 (China Standard Time)

Yes that will work!

Just to break that down: you'll have to modify the hippunfold code that's inside the container. Give it a try and if you still have problems let me know and I'll try to help

jlhanson5 · Answer 13 · Tue Oct 17 2023 01:26:48 GMT+0800 (China Standard Time)

Great, our team will try that and report back.
Thanks!

eva bacas · Answer 14 · Wed Nov 15 2023 03:32:46 GMT+0800 (China Standard Time)

Hi! Hopefully it's ok to continue using this thread, though I think I am having different issues. I am part of Jamie's lab. We modified the code as directed and tested again on our lab's server - with success!

Because we aim to process a Lot of data, I am now running this at our university's supercomputing center. I don't think the setup is significantly different. The main differences are that I am now running the commands in a slurm file and loading packages with lmod.

After an initial successful test on the center's server, we moved on to trying some of the data that we actually want to process (from the ABCD study). And then I started getting some strange errors.

Here's what the data looks like:

[evb32@login1 ~]$ ls hippunfold_test/sub-NDARINVZZZP87KR/*/*
hippunfold_test/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat:
sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_run-01_T1w.json
sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_run-01_T1w.nii.gz

hippunfold_test/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat:
sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_run-01_T1w.json
sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_run-01_T1w.nii.gz

hippunfold_test/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat:
sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_run-01_T1w.json
sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_run-01_T1w.nii.gz

Here's what I'm running:

#!/bin/bash
#SBATCH --account=jhanson
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=28
#SBATCH --cluster=htc
#SBATCH --partition=htc
#SBATCH --time=8:00:00
#SBATCH --job-name=HippunfoldTest1

module load singularity
module load gcc/8.2.0
module load python/anaconda3.9-2021.11

SIFILE=khanlab_hippunfold_latest.sif
RUNFILE=hippunfold/hippunfold/run.py
INFOLD=hippunfold_test
OUTFOLD=hippunfold_test/derivatives

singularity exec -e $SIFILE $RUNFILE $INFOLD $OUTFOLD \
participant -p --cores 8 --modality T1w --keep-going

I pulled the SIF file again today just to be sure it is the latest version. I tested with both Python 3.9 and 3.10 (as the local version of Python - I did not edit the SIF file) and got identical errors.

Here's the output:

[evb32@login1 ~]$ cat slurm-2411258.out 

The following have been reloaded with a version change:
  1) gcc/5.4.0 => gcc/8.2.0

Config file config/snakebids.yml is extended by additional config specified via the command line.
Wildcard constraints in inputs are ignored. (rule: template_xfm_itk2ras)
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Job stats:
job                                       count
--------------------------------------  -------
affine_gii_to_native                         18
all                                           1
archive_work_after_final                      3
calculate_curvature_from_surface1             6
calculate_curvature_from_surface2             6
calculate_gyrification1                       6
calculate_gyrification2                       6
calculate_surface_area1                       6
calculate_surface_area2                       6
calculate_thickness_from_surface1             6
calculate_thickness_from_surface2             6
combine_tissue_subfield_labels_corobl         6
compose_warps_native_to_unfold               12
compose_warps_unfold_to_crop_native          12
correct_nan_vertices1                        18
correct_nan_vertices2                        18
cp_unfolded_noconstrain                       6
create_dscalar_metric_cifti                  12
create_native_coord_ref                       6
create_native_crop_ref                        6
create_spec_file_dentate                      6
create_spec_file_hipp                         6
create_warps_hipp                             6
expand_unfolded_warps                        12
get_subfield_vols_subj                        3
label_subfields_from_vol_coords_corobl        6
laplace_coords_hipp                          18
merge_lr_spec_file                            6
metric_to_nii                                18
normalize_curvature1                          6
normalize_curvature2                         12
plot_subj_subfields                           3
qc_subfield                                   6
qc_subfield_surf                              6
resample_coords_native_crop                  42
resample_native_to_crop                       6
resample_subfields_native_crop                6
resample_subfields_to_native                  6
resample_unfoldreg_subfields                  6
unfolded_registration                         6
warp_gii_unfold2corobl1                      18
warp_gii_unfold2corobl2                      18
warp_gii_unfoldreg                           18
total                                       406

Select jobs to execute...

[Tue Nov 14 13:54:50 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-R_laplace-hipp.txt
    jobid: 88
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=2YearFollowUpYArm1, dir=AP, hemi=R
    resources: tmpdir=/tmp, time=30

[Tue Nov 14 13:54:50 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-R_laplace-hipp.txt
    jobid: 467
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=4YearFollowUpYArm1, dir=IO, hemi=R
    resources: tmpdir=/tmp, time=30

[Tue Nov 14 13:54:50 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-L_laplace-hipp.txt
    jobid: 214
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=2YearFollowUpYArm1, dir=IO, hemi=L
    resources: tmpdir=/tmp, time=30

[Tue Nov 14 13:54:50 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-R_laplace-hipp.txt
    jobid: 218
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=2YearFollowUpYArm1, dir=IO, hemi=R
    resources: tmpdir=/tmp, time=30

[Tue Nov 14 13:54:50 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-L_laplace-hipp.txt
    jobid: 463
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=4YearFollowUpYArm1, dir=IO, hemi=L
    resources: tmpdir=/tmp, time=30

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp9f39wjzk.laplace_coords_withinit.py
/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp6h5jgjmw.laplace_coords_withinit.py

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpjemiz7rz.laplace_coords_withinit.py
/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpg6q5r471.laplace_coords_withinit.py
[Tue Nov 14 13:54:50 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-L_laplace-hipp.txt
    jobid: 262
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=4YearFollowUpYArm1, dir=AP, hemi=L
    resources: tmpdir=/tmp, time=30

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpfqqm2ms4.laplace_coords_withinit.py

[Tue Nov 14 13:54:50 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-R_laplace-hipp.txt
    jobid: 716
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=baselineYear1Arm1, dir=IO, hemi=R
    resources: tmpdir=/tmp, time=30

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpc9q3mw0r.laplace_coords_withinit.py

[Tue Nov 14 13:54:50 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-R_laplace-hipp.txt
    jobid: 337
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=4YearFollowUpYArm1, dir=AP, hemi=R
    resources: tmpdir=/tmp, time=30

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpxl2dhvbw.laplace_coords_withinit.py
/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp44gixji2.laplace_coords_withinit.py
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp44gixji2.laplace_coords_withinit.py", line 7, in <module>
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpfqqm2ms4.laplace_coords_withinit.py", line 7, in <module>
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpjemiz7rz.laplace_coords_withinit.py", line 7, in <module>
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp9f39wjzk.laplace_coords_withinit.py", line 7, in <module>
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpc9q3mw0r.laplace_coords_withinit.py", line 7, in <module>
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp6h5jgjmw.laplace_coords_withinit.py", line 7, in <module>
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpxl2dhvbw.laplace_coords_withinit.py", line 7, in <module>
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpg6q5r471.laplace_coords_withinit.py", line 7, in <module>
    from astropy.convolution import convolve as nan_convolve
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .core import *  # noqa
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from astropy.modeling.core import FittableModel, custom_model
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from . import fitting
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
    from astropy.units import Quantity
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity import *
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    np.asscalar,
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
[Tue Nov 14 13:54:51 2023]
Error in rule laplace_coords_hipp:
    jobid: 463
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-L_laplace-hipp.txt (check log file(s) for error details)

[Tue Nov 14 13:54:51 2023]
Error in rule laplace_coords_hipp:
    jobid: 337
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-R_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpfqqm2ms4.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp44gixji2.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
[Tue Nov 14 13:54:51 2023]
Error in rule laplace_coords_hipp:
    jobid: 218
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-R_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp6h5jgjmw.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
[Tue Nov 14 13:54:51 2023]
Error in rule laplace_coords_hipp:
    jobid: 262
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-AP_hemi-L_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpc9q3mw0r.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
Select jobs to execute...

[Tue Nov 14 13:54:51 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-L_laplace-hipp.txt
    jobid: 12
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=2YearFollowUpYArm1, dir=AP, hemi=L
    resources: tmpdir=/tmp, time=30

[Tue Nov 14 13:54:51 2023]
Error in rule laplace_coords_hipp:
    jobid: 716
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-R_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpxl2dhvbw.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
[Tue Nov 14 13:54:51 2023]
Error in rule laplace_coords_hipp:
    jobid: 467
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-IO_hemi-R_laplace-hipp.txt (check log file(s) for error details)

[Tue Nov 14 13:54:51 2023]
Error in rule laplace_coords_hipp:
    jobid: 214
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-IO_hemi-L_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpjemiz7rz.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpg6q5r471.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
[Tue Nov 14 13:54:51 2023]
Error in rule laplace_coords_hipp:
    jobid: 88
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-R_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp9f39wjzk.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp7l6avkkw.laplace_coords_withinit.py
Select jobs to execute...

[Tue Nov 14 13:54:52 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-L_laplace-hipp.txt
    jobid: 511
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=baselineYear1Arm1, dir=AP, hemi=L
    resources: tmpdir=/tmp, time=30

[Tue Nov 14 13:54:52 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-R_laplace-hipp.txt
    jobid: 594
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=baselineYear1Arm1, dir=PD, hemi=R
    resources: tmpdir=/tmp, time=30

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpa79_v2ih.laplace_coords_withinit.py

[Tue Nov 14 13:54:52 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-L_laplace-hipp.txt
    jobid: 281
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=4YearFollowUpYArm1, dir=PD, hemi=L
    resources: tmpdir=/tmp, time=30

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpc32vj94c.laplace_coords_withinit.py

[Tue Nov 14 13:54:52 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-R_laplace-hipp.txt
    jobid: 96
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=2YearFollowUpYArm1, dir=PD, hemi=R
    resources: tmpdir=/tmp, time=30

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp25g8mcez.laplace_coords_withinit.py

[Tue Nov 14 13:54:52 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-R_laplace-hipp.txt
    jobid: 586
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=baselineYear1Arm1, dir=AP, hemi=R
    resources: tmpdir=/tmp, time=30

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp25_gkmdm.laplace_coords_withinit.py

[Tue Nov 14 13:54:52 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-L_laplace-hipp.txt
    jobid: 712
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=baselineYear1Arm1, dir=IO, hemi=L
    resources: tmpdir=/tmp, time=30

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp7oifmeav.laplace_coords_withinit.py

[Tue Nov 14 13:54:52 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-R_laplace-hipp.txt
    jobid: 345
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=4YearFollowUpYArm1, dir=PD, hemi=R
    resources: tmpdir=/tmp, time=30

/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpr91aor71.laplace_coords_withinit.py
Select jobs to execute...
/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp2i1rmiqk.laplace_coords_withinit.py
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp7l6avkkw.laplace_coords_withinit.py", line 7, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpa79_v2ih.laplace_coords_withinit.py", line 7, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp25g8mcez.laplace_coords_withinit.py", line 7, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpc32vj94c.laplace_coords_withinit.py", line 7, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
Traceback (most recent call last):
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp7oifmeav.laplace_coords_withinit.py", line 7, in <module>
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp25_gkmdm.laplace_coords_withinit.py", line 7, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
    np.asscalar,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
[Tue Nov 14 13:54:52 2023]
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
Error in rule laplace_coords_hipp:
    jobid: 12
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-AP_hemi-L_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp7l6avkkw.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run

[Tue Nov 14 13:54:52 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-L_laplace-hipp.txt
    jobid: 32
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=2YearFollowUpYArm1, dir=PD, hemi=L
    resources: tmpdir=/tmp, time=30

Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpr91aor71.laplace_coords_withinit.py", line 7, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
[Tue Nov 14 13:54:52 2023]
Error in rule laplace_coords_hipp:
    jobid: 511
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-L_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpa79_v2ih.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp2630tfdb.laplace_coords_withinit.py
Select jobs to execute...

[Tue Nov 14 13:54:52 2023]
rule laplace_coords_hipp:
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-L_laplace-hipp.txt
    jobid: 530
    reason: Missing output files: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    wildcards: subject=NDARINVZZZP87KR, session=baselineYear1Arm1, dir=PD, hemi=L
    resources: tmpdir=/tmp, time=30

Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp2i1rmiqk.laplace_coords_withinit.py", line 7, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
/opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpnlknvj09.laplace_coords_withinit.py
[Tue Nov 14 13:54:52 2023]
Error in rule laplace_coords_hipp:
    jobid: 281
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-L_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp25g8mcez.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
[Tue Nov 14 13:54:52 2023]
Error in rule laplace_coords_hipp:
    jobid: 594
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-R_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpc32vj94c.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
[Tue Nov 14 13:54:52 2023]
Error in rule laplace_coords_hipp:
    jobid: 712
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-IO_hemi-L_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpr91aor71.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
[Tue Nov 14 13:54:52 2023]
Error in rule laplace_coords_hipp:
    jobid: 96
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-R_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp25_gkmdm.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
[Tue Nov 14 13:54:52 2023]
Error in rule laplace_coords_hipp:
    jobid: 586
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-AP_hemi-R_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp7oifmeav.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
[Tue Nov 14 13:54:52 2023]
Error in rule laplace_coords_hipp:
    jobid: 345
    input: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_hemi-R_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-R_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-4YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-4YearFollowUpYArm1_dir-PD_hemi-R_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp2i1rmiqk.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/opt/conda/lib/python3.9/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp2630tfdb.laplace_coords_withinit.py", line 7, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
Traceback (most recent call last):
  File "/ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpnlknvj09.laplace_coords_withinit.py", line 7, in <module>
    from astropy.convolution import convolve as nan_convolve
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/__init__.py", line 4, in <module>
    from .core import *  # noqa
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/core.py", line 23, in <module>
    from .utils import (discretize_model, add_kernel_arrays_1D,
  File "/opt/conda/lib/python3.9/site-packages/astropy/convolution/utils.py", line 5, in <module>
    from astropy.modeling.core import FittableModel, custom_model
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/__init__.py", line 10, in <module>
    from . import fitting
  File "/opt/conda/lib/python3.9/site-packages/astropy/modeling/fitting.py", line 39, in <module>
    from astropy.units import Quantity
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/__init__.py", line 17, in <module>
    from .quantity import *
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity.py", line 28, in <module>
    from .quantity_helper import (converters_and_unit, can_have_arbitrary_unit,
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/__init__.py", line 10, in <module>
    from . import helpers, function_helpers
  File "/opt/conda/lib/python3.9/site-packages/astropy/units/quantity_helper/function_helpers.py", line 119, in <module>
    np.asscalar,
  File "/ihome/jhanson/evb32/.local/lib/python3.9/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
[Tue Nov 14 13:54:53 2023]
Error in rule laplace_coords_hipp:
    jobid: 32
    input: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/anat/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/coords/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-2YearFollowUpYArm1/sub-NDARINVZZZP87KR_ses-2YearFollowUpYArm1_dir-PD_hemi-L_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmp2630tfdb.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
[Tue Nov 14 13:54:53 2023]
Error in rule laplace_coords_hipp:
    jobid: 530
    input: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/anat/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_hemi-L_space-corobl_desc-postproc_dseg.nii.gz, work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-init_coords.nii.gz
    output: work/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/coords/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-L_space-corobl_label-hipp_desc-laplace_coords.nii.gz
    log: logs/sub-NDARINVZZZP87KR/ses-baselineYear1Arm1/sub-NDARINVZZZP87KR_ses-baselineYear1Arm1_dir-PD_hemi-L_laplace-hipp.txt (check log file(s) for error details)

RuleException:
CalledProcessError in file /ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk, line 84:
Command 'set -euo pipefail;  /opt/conda/bin/python3 /ihome/jhanson/evb32/hippunfold_test/derivatives/.snakemake/scripts/tmpnlknvj09.laplace_coords_withinit.py' returned non-zero exit status 1.
  File "/ihome/jhanson/evb32/hippunfold/hippunfold/workflow/rules/autotop.smk", line 84, in __rule_laplace_coords_hipp
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 52, in run
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-11-14T135447.858966.snakemake.log

It seems to me like the problem is numpy.asscalar, which is depreciated in the version of numpy that the SIF file is running. But I don't know why that problem did not come up for the other data... so it must be something specific to this data.

Let me know if I can provide any additional information. Thanks so much for your help!

Ali Khan · Answer 15 · Fri Nov 17 2023 01:18:14 GMT+0800 (China Standard Time)

Hi Eva,

Looks like a python dependency issue that might be caused by a python environment outside of the container being picked-up inadvertently. Can you try re-running without the:

module load gcc/8.2.0
module load python/anaconda3.9-2021.11

in your script. Those shouldn't be needed anyways since everything is in the container, all you need is the module for singularity.