Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

# FATAL: while parsing bind path: wrong bind syntax #776

Open
viralp opened this issue Apr 5, 2023 · 4 comments
Open

# FATAL: while parsing bind path: wrong bind syntax #776

viralp opened this issue Apr 5, 2023 · 4 comments

Comments

@viralp
Copy link

viralp commented Apr 5, 2023

Hi

I am getting the following singularity error in stimela.
log-A168_D_Config_reduction_script.txt
log-A168_D_Config_reduction_script-imaging_round_init.txt

2023-04-05 15:58:41 STIMELA.A168 D Config reduction script INFO: ---------------------------------
2023-04-05 15:58:41 STIMELA.A168 D Config reduction script INFO: Stimela version 1.7.6
2023-04-05 15:58:41 STIMELA.A168 D Config reduction script INFO: Running: A168 D Config reduction script
2023-04-05 15:58:41 STIMELA.A168 D Config reduction script INFO: ---------------------------------
2023-04-05 15:58:41 STIMELA.A168 D Config reduction script INFO: Config directory cannot be found. Will create /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345/stimela_parameter_files
2023-04-05 15:58:41 STIMELA.A168 D Config reduction script INFO: Adding cab '/export/home/viralp/singularity_images/stimela_wsclean_1.7.4.sif' (3.1) to recipe, container name 'imaging_round_init'
2023-04-05 15:58:41 STIMELA.imaging_round_init INFO: job started at 2023-04-05 15:58:41.343580
2023-04-05 15:58:41 STIMELA.A168 D Config reduction script INFO: STEP 1 :: imaging_round_init:: Make initial image
2023-04-05 15:58:41 STIMELA INFO: Validating parameters for cab wsclean (stimela/wsclean:1.7.4)
2023-04-05 15:58:41 STIMELA INFO: Parameters validated and saved to /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345/stimela_parameter_files/imaging_round_init-14042593832337616807319211861491.json
2023-04-05 15:58:41 STIMELA.imaging_round_init INFO: Starting container [imaging_round_init-14042593832337616807319211861491]. Timeout set to -1. The container ID is printed below.
# running cd /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345 && singularity run --workdir /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345 --containall  --bind /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345/stimela_parameter_files/imaging_round_init-14042593832337616807319211861491.json:/stimela_mount/configfile:ro --bind /users/vparekh/venv3/lib/python3.8/site-packages/stimela/cargo/cab/wsclean/src:/stimela_mount/code:ro --bind /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345/passwd:/etc/passwd:rw --bind /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345/group:/etc/group:rw --bind /users/vparekh/venv3/bin/stimela_runscript:/singularity:ro --bind /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/msdir:/stimela_mount/msdir:rw --bind /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/input:/stimela_mount/input:ro --bind /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/output_A168_relic_target:/stimela_mount/output:rw --bind /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/output_A168_relic_target/tmp:/stimela_mount/output/tmp:rw /export/home/viralp/singularity_images/stimela_wsclean_1.7.4.sif /singularity
# FATAL:   while parsing bind path: wrong bind syntax: /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345/passwd:/etc/passwd:rw
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR: cd /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345 && singularity run --workdir /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345 --containall returns error code 255
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR: job failed at 2023-04-05 15:58:41.426185 after 0:00:00.082605
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR: Traceback (most recent call last):
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR:   File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/recipe.py", line 713, in run
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR:     job.run_job()
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR:   File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/recipe.py", line 425, in run_job
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR:     self.job.run(output_wrangler=self.apply_output_wranglers)
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR:   File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/singularity.py", line 123, in run
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR:     utils.xrun(f"cd {self.execdir} && singularity run --workdir {self.execdir} --containall",
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR:   File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/utils/xrun_poll.py", line 227, in xrun
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR:     raise StimelaCabRuntimeError("{} returns error code {}".format(command_name, status))
2023-04-05 15:58:41 STIMELA.imaging_round_init ERROR: stimela.utils.StimelaCabRuntimeError: cd /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345 && singularity run --workdir /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345 --containall returns error code 255
2023-04-05 15:58:41 STIMELA.A168 D Config reduction script INFO: Completed jobs : []
2023-04-05 15:58:41 STIMELA.A168 D Config reduction script INFO: Remaining jobs : []
2023-04-05 15:58:41 STIMELA.A168 D Config reduction script INFO: Saving pipeline information in .last_A168_D_Config_reduction_script.json
Traceback (most recent call last):
  File "/users/vparekh/venv3/bin/stimela", line 11, in <module>
    main.main([a for a in sys.argv[1:]])
  File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/main.py", line 409, in main
    _cmd(argv)
  File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/main.py", line 199, in run
    exec(stdr.read(), _globals)
  File "<string>", line 69, in <module>
  File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/recipe.py", line 764, in run
    raise PipelineException(exc, self.completed, job, self.remaining) from None
stimela.exceptions.PipelineException: Job 'imaging_round_init:: Make initial image' failed: cd /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345 && singularity run --workdir /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16807319211808345 --containall returns error code 255

stimela_script_vla.txt
@o-smirnov
Copy link
Collaborator

This looks to be the real error, singularity doesn't understand the command line Stimela forms up:

while parsing bind path: wrong bind syntax:

@viralp what version of singularity is on this box? I presume it's either too old (not our problem then) or too new (in which case it is...)

@viralp
Copy link
Author

viralp commented Apr 10, 2023

Seems to be old version.

singularity version 3.6.1+3-gb94fa3544

@viralp
Copy link
Author

viralp commented Apr 18, 2023

Ok singularity has been updated to 3.11. Now getting the following error

#  == Constructing PSF ==
# Detected 62.4 GB of system memory, usage not limited.
# Opening reordered part 0 spw 0 for /stimela_mount/msdir/A168_VLA_L_D_targets.ms
# Determining min and max w & theoretical beam size... Traceback (most recent call last):
#   File "/stimela_mount/code/run.py", line 69, in <module>
#     subprocess.check_call(shlex.split(_runc))
#   File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
#     raise CalledProcessError(retcode, cmd)
# subprocess.CalledProcessError: Command '['wsclean', '-name', '/stimela_mount/output/17A-187.sb33657704.eb33666723.57871.671216585644init', '-j', '32', '-mem', '100', '-weight', 'briggs', '-0.5', '-size', '6000', '6000', '-scale', '0.06arcsec', '-channels-out', '32', '-no-small-inversion', '-pol', 'I', '-data-column', 'DATA', '-niter', '2000000', '-auto-threshold', '0.5', '-auto-mask', '5.0', '-mgain', '0.9', '-join-channels', '-fit-spectral-pol', '4', '-nmiter', '20', '/stimela_mount/msdir/A168_VLA_L_D_targets.ms']' died with <Signals.SIGKILL: 9>.
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR: cd /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16818497755035388 && singularity run --workdir /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16818497755035388 --containall returns error code 1
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR: job failed at 2023-04-18 14:30:08.140287 after 0:00:32.468797
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR: Traceback (most recent call last):
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR:   File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/recipe.py", line 713, in run
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR:     job.run_job()
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR:   File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/recipe.py", line 425, in run_job
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR:     self.job.run(output_wrangler=self.apply_output_wranglers)
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR:   File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/singularity.py", line 123, in run
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR:     utils.xrun(f"cd {self.execdir} && singularity run --workdir {self.execdir} --containall",
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR:   File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/utils/xrun_poll.py", line 227, in xrun
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR:     raise StimelaCabRuntimeError("{} returns error code {}".format(command_name, status))
2023-04-18 14:30:08 STIMELA.imaging_round_init ERROR: stimela.utils.StimelaCabRuntimeError: cd /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16818497755035388 && singularity run --workdir /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16818497755035388 --containall returns error code 1
2023-04-18 14:30:08 STIMELA.A168 Config reduction script (Sequential) INFO: Completed jobs : []
2023-04-18 14:30:08 STIMELA.A168 Config reduction script (Sequential) INFO: Remaining jobs : []
2023-04-18 14:30:08 STIMELA.A168 Config reduction script (Sequential) INFO: Saving pipeline information in .last_A168_Config_reduction_script__Sequential_.json
Traceback (most recent call last):
  File "/users/vparekh/venv3/bin/stimela", line 11, in <module>
    main.main([a for a in sys.argv[1:]])
  File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/main.py", line 409, in main
    _cmd(argv)
  File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/main.py", line 199, in run
    exec(stdr.read(), _globals)
  File "<string>", line 76, in <module>
  File "/users/vparekh/venv3/lib/python3.8/site-packages/stimela/recipe.py", line 764, in run
    raise PipelineException(exc, self.completed, job, self.remaining) from None
stimela.exceptions.PipelineException: Job 'imaging_round_init:: Make initial image' failed: cd /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16818497755035388 && singularity run --workdir /export/home/viralp/A168/17A-187.sb33657704.eb33666723.57871.671216585644_d/test2/stimela/.stimela_workdir-16818497755035388 --containall returns error code 1

@bennahugo
Copy link
Collaborator

bennahugo commented Apr 19, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@viralp @o-smirnov @bennahugo