My app is a bash script that runs tesseract on GNU parallel. The data I need to process is to the tune of 50GB. It's too slow if I do it one VM. I need the power of cluster computing but I don't want to set up multiples VMs myself, instead I just want to launch my APP (along with the data files) on Google cluster (Kubernetes?). I don't have much clarity about these concepts. If someone can guide that would be great.
Cluster, Kubernetes, Docker. What to choose for my app?
104 views Asked by Naveed At
1
There are 1 answers
Related Questions in DOCKER
- sqlplus myusername/mypassword@ORCL not working with Oracle on Docker
- Golang == Error: OCI runtime create failed: unable to start container process: exec: "./bin": stat ./bin: no such file or directory: unknown
- Only the first SQL script gets executed inside Docker Postgres container
- Retrieve the Dockerfile configuration from the Kubernetes and also change container Java parameter?
- Polars with Rust: Out of Memory Error when Processing Large Dataset in Docker Using Streaming
- Compiling eBPF program in Docker fails due to missing '__u64' type
- AttributeError: module 'numba' has no attribute 'generated_jit'
- Phoenix in a docker dev environment - generated code can't be saved from VSCode
- Docker on Multipass VMs: Connecting worker nodes to swarm results in rcp error
- Facing error in creating image of my react+vite project . Dockerfile error
- NextJS Docker build fails: fetch failed ECONNREFUSED
- Docker container unable to make HTTPS requests to external API
- Failed to connect to your instance after deploying mern app on aws ec2 instance when i try to access frontend
- Connecting to Postgres running in a Docker container using psql
- Can't connect to local postgresql server from my docker container
Related Questions in KUBERNETES
- Golang == Error: OCI runtime create failed: unable to start container process: exec: "./bin": stat ./bin: no such file or directory: unknown
- I can't create a pod in minikube on windows
- Oracle setting up on k8s cluster using helm charts enterprise edition
- Retrieve the Dockerfile configuration from the Kubernetes and also change container Java parameter?
- Summarize pods not running, by Namespace and Reason - I'm having trouble finding the reason
- How to get Java running parameters from Spring Boot running inside container in pod where no ps exist
- How do we configure prometheus server to scrape metrics from a pod with Istio sidecar proxy?
- In rke kube-proxy pod is not present
- problem with edge server registration in Eureka
- Unable to Access Kubernetes LoadBalancer Service from Local Device Outside Cluster
- Kubernetes cluster on GCE connection refused error
- Based on my experience, I've outlined the Kubernetes request flow. Could someone please add or highlight any points I might have overlooked?
- how to define StackGres helm chart "restapi" values to use internal LoadBalancer - AWS EKS
- Python3.11 can't open file [Errno 2] No such file or directory
- Cannot find remote pod service - SERVICE_UNAVAILABLE
Related Questions in CLUSTER-COMPUTING
- How to Socket.IO Multithreading on a Raspberry Pi?
- Snakemake remote rules re-read config file?
- Why can CPU memory be specified and allocated during instance creation but not GPU memory on the cloud?
- Make a Cluster without using MongoDB Atlas
- Why don't run at multiple thread?
- Imports failing with workaround in Google Dataproc Cluster Notebooks
- Galera Cluster: 3 Node Cluster, One Node does not reconnect after rebooting
- How to install gromacs on gcp HPC
- Slurm - How to run a list of jobs n by n?
- Setup Slurm partition for only interactive jobs
- makeCluster fails with remote server Ubuntu
- Creating a cluster with two laptops (mac)
- How to configure express-fileupload with cluster?
- problem in configuring dataproc cluster from GCP Console since Friday (1 february 2024)
- Kubernetes cluster refused to connect google cloud
Related Questions in CONTAINER-IMAGE
- Kubernetes controller-manager and scheduler restarting multiple times
- Does a container image "digest" uniquely identify that particular container such that any change to the actual content will change the digest?
- Podman push image fails with tls certification verification error
- How do we map a container image layer to its source Dockerfile command?
- sudo execution fails in do_pull_image() of .bb file
- Getting spring boot build error when trying to create jar file using docker container configurations in application.properties
- Kafka UI not able to connect to Broker
- Invoking Java from container - directly or shell
- How to share jar layers between diffrent microservices?
- Verify container images at push to local registry
- Container image - What is the purpose of using the phrase "Container image"?
- installing managed Docker (network) plugin from an exported Docker image file?
- Debugging AWS Lambda (Python) Locally in VS Code with out using SAM-CLI
- Receiving errors when pushing container image to aws lambda
- After deleting and re-creating a GitHub repository, the default token can no longer publish container images
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Might be a challenge to learn all the container orchestration details from scratch when you are just concerned about this one use case.
While GNU Parrellel is nice on a single machine, there don't seem to be many starter kits for using it in distributed mode in the cloud.
I would consider google dataflow rather than spining up a K8S cluster. It allocates and cleans up easily and lets you avoid managing VMs and learning an orchestration framework.