Can anyone illustrate the difference between deploying the Spark Operator with Argoflow & Apache Spark with Knative. Which route will help to achieve a pure serverless application stack over Kubernretes.
Argo Workflow + Spark Operator Or Spark with Knative leads to Server less Deployment
316 views Asked by Aman Raparia At
1
There are 1 answers
Related Questions in APACHE-SPARK
- Getting error while running spark-shell on my system; pyspark is running fine
- ingesting high volume small size files in azure databricks
- Spark load all partions at once
- Databricks Delta table / Compute job
- Autocomplete not working for apache spark in java vscode
- How to overwrite a single partition in Snowflake when using Spark connector
- Parse multiple record type fixedlength file with beanio gives oom and timeout error for 10GB data file
- includeExistingFiles: false does not work in Databricks Autoloader
- Spark connectors from Azure Databricks to Snowflake using AzureAD login
- SparkException: Task failed while writing rows, caused by Futures timed out
- Configuring Apache Spark's MemoryStream to simulate Kafka stream
- Databricks can't find a csv file inside a wheel I installed when running from a Databricks Notebook
- Add unique id to rows in batches in Pyspark dataframe
- Does Spark Dynamic Allocation depend on external shuffle service to work well?
- Does Spark structured streaming support chained flatMapGroupsWithState by different key?
Related Questions in KUBERNETES
- Golang == Error: OCI runtime create failed: unable to start container process: exec: "./bin": stat ./bin: no such file or directory: unknown
- I can't create a pod in minikube on windows
- Oracle setting up on k8s cluster using helm charts enterprise edition
- Retrieve the Dockerfile configuration from the Kubernetes and also change container Java parameter?
- Summarize pods not running, by Namespace and Reason - I'm having trouble finding the reason
- How to get Java running parameters from Spring Boot running inside container in pod where no ps exist
- How do we configure prometheus server to scrape metrics from a pod with Istio sidecar proxy?
- In rke kube-proxy pod is not present
- problem with edge server registration in Eureka
- Unable to Access Kubernetes LoadBalancer Service from Local Device Outside Cluster
- Kubernetes cluster on GCE connection refused error
- Based on my experience, I've outlined the Kubernetes request flow. Could someone please add or highlight any points I might have overlooked?
- how to define StackGres helm chart "restapi" values to use internal LoadBalancer - AWS EKS
- Python3.11 can't open file [Errno 2] No such file or directory
- Cannot find remote pod service - SERVICE_UNAVAILABLE
Related Questions in GOOGLE-KUBERNETES-ENGINE
- Golang == Error: OCI runtime create failed: unable to start container process: exec: "./bin": stat ./bin: no such file or directory: unknown
- Kubectl command throws error when executed from python script but manual execution works fine
- Unable to add TLS certificate to GKE from Google Secret Manager
- GKE Clusters no indication within metrics or logs after failure
- Getting connection refused to Private GKE Internal point. Autopilot private cluster
- Can I have the Google managed service range on a standard gke cluster created with Terraform- non auto-pilot
- Configure Lens with GKE cluster - gke_gcloud_auth_plugin issue
- Having issues joining my kubernetes worker noed to a controller node
- How to deploy airflow in kubernetes cluster that uses istio
- GKE cluster shutdown
- Artifactory 404 error from virtual repository where docker pulls fine but crictl does not
- Scraping from Mexico in GCP EKG?
- not able to connect via cloud sql proxy
- SQL connection throws error when adding DistributedSession, SessionMiddleware
- CronJob pod repeats pending forever even after deleting it
Related Questions in KNATIVE
- How do I configure multi-tenant Kafka-backed Knative channels in an Openshift?
- Knative's support for the Socket protocol
- Is it possible to send KNative events using any kind of SDK in java/kotlin without relying on Http?
- How to mount NFS in Knative v0.23?
- Disabling proxy protocol for the in-cluster communication in Istio
- docker desktop & knative & knative func
- Knative Serving just returns default Apache Webpage
- How should I add HTTP authentication to KNative services?
- Trying to create a Domain Claim in a AKS Cluster using Knative API
- knative domain mapping is not working properly. error: 111 occured. how to use?
- stream terminated by RST_STREAM with error code: PROTOCOL_ERROR
- Trying to request a Knative Autoscale Settings update
- AWS SQS and Knative function
- Slow Knative service creation times
- knative eventing with kafka is sending requests again and again
Related Questions in ARGO
- Couldn't find the image pushed in dockerhub using buildkit and argo workflow
- argo workflow - error evaluating expression
- In ArgoCD is it possible to customize the text displayed in the deployment node, for example?
- ArgoCD overriding helm releaseName gives an error
- Get a list of argo workflows which have not finished
- timezone in argo workflow
- argocd-vault-plugin 'generate' section is failing on me with a weird error
- Remove element from Kubernetes object array with Argo Events
- How To Configure Argo-Workflows with SSO with Helm Values File
- ArgoCD: Multiple sources for a helm chart
- Argoworkflow pass argo step parameter as null if the step is skipped
- Patch argo-server authentication | kubectl gives "error: unknown flag: --auth-mode"
- argo workflow set array of integer as container env
- integrate Argo rollouts dashboard into react application
- Argo Rollout AnalysisRun datadog with incomplete options
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
I'm not sure what configuration you're imagining for Apache Spark + Knative. Knative is intended to provide developers a serverless deployment and management experience, but AIUI Spark is an independent project which does not specifically target Kubernetes execution.
Similarly, it looks like Argoflow aims to simplify deployment of Kubeflow (ML training) applications specifically, whereas Spark is a general-purpose data processing framework. Your question is a bit like asking "what's better for getting around, a Tesla or a Cessna"? Without knowing where you're going, it's a bit hard to make a recommendation.