2_contig_assembly Error

[neelanjaya]

When running quackers errors are being shown in step 2 (2_contig_assembly). The clean_reads_reconcile.sh script and clean_reads.sh script are showing 'perl error' and 'float split' errors. This may also be related to sam reorganization or sorting (as the error file says 'sam_clean.py' is being run). I have attached part of the error below.

Finding the cause of this issue is difficult to trace, as the generated txt files are not a robust checkpoint.

A feature that may be useful is a check in before moving to the next step, and ending if the files were not generated. This checkpointing can be written all to one file rather than checkpoint files.

2_contig_assembly reconcile error:
[bam_sort_core] merging from 24 files and 1 in-memory blocks... Traceback (most recent call last): File "/quackers_pipe/scripts/contig_reconcile.py", line 184, in <module> sam_hits_dict, unique_hosts = sort_samfiles(sam_score_file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/quackers_pipe/scripts/contig_reconcile.py", line 101, in sort_samfiles old_score = float(old_hit.split("|")[1]) ^^^^^^^^^^^^^ AttributeError: 'float' object has no attribute 'split' Traceback (most recent call last): File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 573, in _build_master ws.require(__requires__) File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 891, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 782, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (numpy 2.0.0 (/usr/local/lib/python3.10/dist-packages), Requirement.parse('numpy<2,>=1.22.4'), {'pandas'})

2_contig_assembly clean reads error:
`/quackers_tools/SPAdes/share/spades/spades_pipeline/support.py:488: SyntaxWarning: invalid escape sequence '\d'
return [atoi(c) for c in re.split("(\d+)", text)]
Process Process-5:
Traceback (most recent call last):
File "/opt/conda/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/opt/conda/lib/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/scratch/j/jparkin/neelnjay/toolbox/quackers/quackers-1.0.1/MetaPro_utilities.py", line 294, in create_and_launch_v2
sp.check_output(["sh", job_path])#, stderr = sp.STDOUT)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.12/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sh', '/scratch/j/jparkin/neelnjay/hlhs/quackers_101_output/2/2_contig_assemble/assemble_p.sh']' returned non-zero exit status 2.
...

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = "C.UTF-8",
LANG = "en_CA.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Error, fewer reads in file specified with -1 than in file specified with -2
terminate called after throwing an instance of 'int'
Aborted (core dumped)
(ERR): bowtie2-align exited with value 134
Process Process-8:
Traceback (most recent call last):
File "/opt/conda/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/opt/conda/lib/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/scratch/j/jparkin/neelnjay/toolbox/quackers/quackers-1.0.1/MetaPro_utilities.py", line 294, in create_and_launch_v2
sp.check_output(["sh", job_path])#, stderr = sp.STDOUT)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.12/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sh', '/scratch/j/jparkin/neelnjay/hlhs/quackers_101_output/2/2_contig_assemble/clean_reads.sh']' returned non-zero exit status 134.
[E::sam_parse1] SEQ and QUAL are of different length
[W::sam_read1_sam] Parse error at line 9340514
samtools view: error reading file "/scratch/j/jparkin/neelnjay/hlhs/quackers_101_output/2/2_contig_assemble/data/raw_bt2_out.sam"
Traceback (most recent call last):
File "/quackers_pipe/scripts/contig_reconcile.py", line 184, in
sam_hits_dict, unique_hosts = sort_samfiles(sam_score_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/quackers_pipe/scripts/contig_reconcile.py", line 87, in sort_samfiles
with open(sam_path, "r") as sam_in:
^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/scratch/j/jparkin/neelnjay/hlhs/quackers_101_output/2/2_contig_assemble/data/score_bt2.out'
[E::hts_open_format] Failed to open file "/scratch/j/jparkin/neelnjay/hlhs/quackers_101_output/2/2_contig_assemble/data/bt2_sort.bam" : No such file or directory
ERROR: fail to open index BAM file '/scratch/j/jparkin/neelnjay/hlhs/quackers_101_output/2/2_contig_assemble/data/bt2_sort.bam'
Traceback (most recent call last):
File "/quackers_pipe/modded_scripts/concoct_coverage_table.py", line 100, in
generate_input_table(args.bedfile, args.bamfiles, samplenames=samplenames)
File "/quackers_pipe/modded_scripts/concoct_coverage_table.py", line 46, in generate_input_table
raise Exception('Error with running samtools bedcov')`