I was trying to run some whole genome sequencing samples on google cloud using dsub. The dsub commands work ok for some samples, but not others. I have tried reducing the number of parallel threads, increasing the memory and disk, but it still fails. Since each run takes about 2 days, the trial and error approach is pretty expensive! Any help/tips would be highly appreciated!
My command is:
dsub \
--project "${MY_PROJECT}" \
--zones "us-central1-a" \
--logging "${LOGGING}" \
--vars-include-wildcards \
--disk-size 800 \
--min-ram 60 \
--image "us.gcr.io/xxx-yyy-zzz/data" \
--tasks "${SCRIPT_DIR}"/tBOWTIE2.tsv \
--command 'bismark --bowtie2 --bam --parallel 2 "${GENOME_REFERENCE}" -1 "${INPUT_FORWARD}" -2 "${INPUT_REVERSE}" -o "${OUTPUT_DIR}"' \
--wait
The dstat command with '--full' option shows the error as:
status: FAILURE
status-detail: "11: Docker run failed"
The last line in the log file, on google cloud, just states "(exit status 141)".
many thanks!