error in visualization and DMR analysis

[GabrieleRigano99]

Hi @jkimlab ,
I am having some issues with the visualization step and the DMR analysis:
I have done the previous steps without using the docker image since the program failed at the calling step (like it happened in the other issue #2 ) and I proceeded with docker starting from the GMA step. I tried running the analysis either with --bsmooth and without it, but it dies with these errors

/msPIPE/bin/vis_script/GMA.Circos.R /work_dir/RawData/Analysis/Ideogram.bed /work_dir/RawData/Analysis/GC/ /work_dir/RawData/Analysis/GC/Circos_GC.CpG_UMRs_LMRs.pdf

*** There's a problem executing the upper command.
*** check logfile > /work_dir/RawData/Analysis/logs/log.GC.Circos
Rscript /msPIPE/bin/vis_script/genomic_context_levels.R /work_dir/RawData/Analysis/PG/methylation.Genomic_Context.CpG.txt PG /work_dir/RawData/Analysis/PG/Genomic_Context_CpG.pdf 1> /work_dir/RawData/Analysis/PG/Avg_Genomic_Context_CpG.txt
Error: unable to open file or unable to determine types for file /work_dir/RawData/Analysis/DMR/GC.LG/reform.DMC_q0.5.bed

• Please ensure that your file is TAB delimited (e.g., cat -t FILE).
• Also ensure that your file has integer chromosome coordinates in the
  expected columns (e.g., cols 2 and 3 for BED).
  UPPER STEPS ARE NOT COMPLETED.
  ERR: problem in Visualization step

DMR Analysis ...------------------------------------------------------------
DMR .. start
GC, LG

Rscript /work_dir/RawData/Analysis/DMR/GC.LG/methylkit/run_methylKit.R
$ /msPIPE/bin/script/getDMC.py /work_dir/RawData/Analysis/DMR/GC.LG/methylkit/CpG_united_filtered_meth.txt /work_dir/RawData/Analysis/DMR/GC.LG/methylkit/CpG_united_filtered_raw_DMBs.txt 0.5 0,0,0,0,1,1,1 /work_dir/RawData/Analysis/DMR/GC.LG/DMC_q0.5.bed GC LG
$ bedtools intersect -wa -wb -a /work_dir/RawData/Analysis/annotations/promoter.bed -b /work_dir/RawData/Analysis/DMR/GC.LG/reform.DMC_q0.5.bed > /work_dir/RawData/Analysis/DMR/GC.LG/intersection.DMC2Promoter.txt

*** There's a problem executing the upper command.
*** check logfile > stdout

$ cut -f 4 /work_dir/RawData/Analysis/DMR/GC.LG/intersection.DMC2Promoter.txt | cut -d ';' -f1| cut -d ':' -f2 |sort -u > /work_dir/RawData/Analysis/DMR/GC.LG/DMC_genelist.txt
$ Rscript /msPIPE/bin/GMA/gProfiler.R /work_dir/RawData/Analysis/DMR/GC.LG/DMC_genelist.txt danRer11 /work_dir/RawData/Analysis/DMR/GC.LG/DMC_gene

*** There's a problem executing the upper command.
*** check logfile > /work_dir/RawData/Analysis/DMR/GC.LG/logs/log.gprofiler.txt

$ /msPIPE/bin/script/getStat.pl /work_dir/RawData/Analysis/DMR/GC.LG/intersection.DMC2Promoter.txt /work_dir/RawData/Analysis/DMR/GC.LG
$ /msPIPE/bin/script/getDMR_500bp.pl /work_dir/RawData/Analysis/DMR/GC.LG/methylkit/CpG_united_filtered_raw_DMBs.txt 0.5 1> /work_dir/RawData/Analysis/DMR/GC.LG/DMR_0.5.bed

output with --bsmooth
Rscript /msPIPE/bin/script/BSmooth_DMR.R /work_dir/RawData/Analysis/DMR/GC.LG/bsmooth_param.txt GC,LG /work_dir/RawData/Analysis 0.5 1> /work_dir/RawData/Analysis/DMR/GC.LG/logs/log.DMR_BSmooth.txt
$ /msPIPE/bin/script/BSmooth_reform.py /work_dir/RawData/Analysis/DMR/GC.LG/DMR_q0.5.bed 1> /work_dir/RawData/Analysis/DMR/GC.LG/reform.DMR_q0.5.bed
$ bedtools intersect -wa -wb -a /work_dir/RawData/Analysis/annotations/promoter.bed -b /work_dir/RawData/Analysis/DMR/GC.LG/reform.DMR_q0.5.bed > /work_dir/RawData/Analysis/DMR/GC.LG/intersection.DMR2Promoter.txt

*** There's a problem executing the upper command.
*** check logfile > stdout

$ cut -f 4 /work_dir/RawData/Analysis/DMR/GC.LG/intersection.DMR2Promoter.txt | cut -d ';' -f1| cut -d ':' -f2 |sort -u > /work_dir/RawData/Analysis/DMR/GC.LG/DMR_genelist.txt
$ Rscript /msPIPE/bin/GMA/gProfiler.R /work_dir/RawData/Analysis/DMR/GC.LG/DMR_genelist.txt danRer11 /work_dir/RawData/Analysis/DMR/GC.LG/DMR_gene

*** There's a problem executing the upper command.
*** check logfile > /work_dir/RawData/Analysis/DMR/GC.LG/logs/log.gprofiler.txt
My guess is that there may be an error in the bed files, either in the normal one (DMC_q0.5.bed) and in the reformed one (reform.DMC_q0.5.bed), they indeed look identical:
NORMAL
chr start end methyl(per)_GC methyl(per)_LG q_val meth_diff
chr1 56555134 56555135 96.90346083788707 98.77750611246944 0.335115544128091 1.87404527458237
chr1 58629922 58629923 96.93486590038314 98.9766081871345 0.323775529940981 2.04174228675137

REFORMED
chr start end methyl(per)_LG methyl(per)_GC q_val meth_diff
chr1 56555134 56555135 98.77750611246944 96.90346083788707 0.335115544128091 1.87404527458237
chr1 58629922 58629923 98.9766081871345 96.93486590038314 0.323775529940981 2.04174228675137

It would be a pity to stop here the analysis, could you help me out please?

Here there are the commands I have launched so far:
~/Softwares/msPIPE/msPIPE.py --param RawData/DMR_CONFIG.conf --out RawData/ --core 24 (this was the first one out of docker)

docker run -v /home/enrico/Projects/domenico/methyl_zebra/RawData/:/msPIPE/data:ro -v /home/enrico/Projects/domenico/methyl_zebra/RawData/:/msPIPE/reference -v /home/enrico/Projects/domenico/methyl_zebra/:/work_dir/ jkimlab/mspipe:latest msPIPE.py -p DMR_CONFIG.conf -o RawData -c 24 --bsmooth --skip_mapping --skip_calling --calling_data RawData/methylCALL

docker run -v /home/enrico/Projects/domenico/methyl_zebra/RawData/:/msPIPE/data:ro -v /home/enrico/Projects/domenico/methyl_zebra/RawData/:/msPIPE/reference -v /home/enrico/Projects/domenico/methyl_zebra/:/work_dir/ jkimlab/mspipe:latest msPIPE.py -p DMR_CONFIG.conf -o RawData -c 24 --skip_mapping --skip_calling --skip_GMA --calling_data RawData/methylCALL

docker run -v /home/enrico/Projects/domenico/methyl_zebra/RawData/:/msPIPE/data:ro -v /home/enrico/Projects/domenico/methyl_zebra/RawData/:/msPIPE/reference -v /home/enrico/Projects/domenico/methyl_zebra/:/work_dir/ jkimlab/mspipe:latest msPIPE.py -p DMR_CONFIG.conf -o RawData -c 24 --bsmooth --skip_mapping --skip_calling --calling_data RawData/methylCALL

Thanks for the super useful tool!