When using AnnotationHub in parallel with job schedular (OpenGridEngine and Slurm), whether a job can be calculated was stochastical and some jobs were failed with the error attempt to write a readonly database.
I don't know why but since some jobs can be calculated,
by performing the job queue, again and again, I could finally get the output.
As an alternative approach, by not using the job schedular,
I could perform the calculation without any error,
but it takes an enormous amount of time, so I want to know the reason for the error.
I wrote an Snakefile as follows:
rule all:
input:
expand('{id}.RData', id=list(range(24)))
rule ah:
output:
'{id}.RData'
log:
'logs/{id}.log'
shell:
'Rscript ah.R {id} >& {log}'wrote the R-script (ah.R) as follows:
outfile = commandArgs(trailingOnly=TRUE)[1]
library("AnnotationHub")
ah = AnnotationHub()
id = names(ah)[sample(length(ah), 1)]
out = ah[[id]]
save(out, file=outfile)and finally performed the snakemake workflow as follows:
# OpenGridEngine
snakemake -j 24 --cluster "qsub -S /bin/bash"
or
# Slurm
snakemake -j 24 --cluster sbatch
The error message was as follows:
Loading required package: BiocFileCache
Loading required package: dbplyr
snapshotDate(): 2019-10-29
downloading 1 resources
retrieving 1 resource
Error: failed to load resource
name: AH39322
title: E031-H3K23me2.imputed.pval.signal.bigwig
reason: 1 resources failed to download
In addition: Warning messages:
1: Couldn't set cache size: attempt to write a readonly database
Use `cache_size` = NULL to turn off this warning.
2: Couldn't set synchronous mode: attempt to write a readonly database
Use `synchronous` = NULL to turn off this warning.
3: download failed
hub path: ‘https://annotationhub.bioconductor.org/fetch/44762’
cache resource: ‘AH39322 : 44762’
reason: attempt to write a readonly database
Execution halted
When using AnnotationHub in parallel with job schedular (OpenGridEngine and Slurm), whether a job can be calculated was stochastical and some jobs were failed with the error
attempt to write a readonly database.I don't know why but since some jobs can be calculated,
by performing the job queue, again and again, I could finally get the output.
As an alternative approach, by not using the job schedular,
I could perform the calculation without any error,
but it takes an enormous amount of time, so I want to know the reason for the error.
I wrote an Snakefile as follows:
wrote the R-script (ah.R) as follows:
and finally performed the snakemake workflow as follows:
or
The error message was as follows: