I would like to use a bash script and qsub to run 30-40 python programs at once. Each python program reads and searches through the same set of files (~400 total) for a set of sequences. Is there a problem where multiple python programs could be trying to read from the same file? If so, what are the consequences?
Can multiple qsub submissions read the same group of files?
105 views Asked by Patrickc01 At
1
There are 1 answers
Related Questions in QSUB
- Interactive jobs via qsh not working in sge
- Getting info on and requesting resources with qsub and slurm
- Pass a variable that contains a comma as a -v option to qsub
- Is there any way by which we can save the output and error files of a PBS script with current time as filename?
- SGE batch file not reading .bashrc and conda command not found
- Batch script not loading modules correctly
- Is it possible to see qsub job scripts and command line options etc. that will be issued by Snakemake in the dry run mode?
- How to print to a common file with qsub arrays?
- I try to run an executable file on a cluster which is compiled with cpp. But I meet some erros just like below
- Passing a qsub command thru ssh2_exec in PHP
- maximum memory for batch jobs with qsub
- Using `snakemake --cluster` without SLURM
- Qsub script - Unable to run job: Script length does not match declared length
- R script works perfectly with Rscript but fails once sent with SGE qsub - problem loading packages
- How to pass parameter expansions into qsub?
Related Questions in SAMETIME
- how to connect 2 flutter app into a single drf api
- set wget to download 10 file at same time
- Serial Port Read and Write with Python in same time
- Two Lslidar(Leishen) connected with One Embedded Board(Jetson Xavier)`s possible?
- Lotus Sametime Embedded - Window Format
- UWP - can't interact with localhost IBM Sametime
- My Threading Chat Server isn't work in parallel . Python 2.7
- Creating common database connection for sevaral forms using qt c++
- Sametime install on ubuntu giving error
- javafx - multiple scenes in same stage
- Save image on Clipboard, and can paste to textview on other application
- OpenCV: Detect face using Haar Cascades and record video at the same time
- Can multiple qsub submissions read the same group of files?
- Sametime plugin development on mac
- In Domino, can we find out if a particular person in names field is online on sametime by using JAVA or Lotusscript?
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Popular Tags
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
There are no problems with multiple jobs reading from the same files that are imposed by Torque. (I think that statement is likely to be true for any resource manager / scheduler.)
The main issue I can imagine would be your file system's performance and whether or not it can keep up with potentially concurrent accesses to those files.