I am getting the error Set current project to spark-parent (in build file:/C:/cygwin64/spark-current/spark-2.1.1/) while trying to build spark. Is there an option "-Dcurrent" or some sbt switch that I can set to facilitate this or do I need to touch a build file? I have my environment variable at the proper location for %SPARK_HOME% so that's not the problem. I'm also working on a windows system but am doing it with cygwin.
Set current project in sbt - spark build issue
824 views Asked by uh_big_mike_boi At
1
There are 1 answers
Related Questions in APACHE-SPARK
- Getting error while running spark-shell on my system; pyspark is running fine
- ingesting high volume small size files in azure databricks
- Spark load all partions at once
- Databricks Delta table / Compute job
- Autocomplete not working for apache spark in java vscode
- How to overwrite a single partition in Snowflake when using Spark connector
- Parse multiple record type fixedlength file with beanio gives oom and timeout error for 10GB data file
- includeExistingFiles: false does not work in Databricks Autoloader
- Spark connectors from Azure Databricks to Snowflake using AzureAD login
- SparkException: Task failed while writing rows, caused by Futures timed out
- Configuring Apache Spark's MemoryStream to simulate Kafka stream
- Databricks can't find a csv file inside a wheel I installed when running from a Databricks Notebook
- Add unique id to rows in batches in Pyspark dataframe
- Does Spark Dynamic Allocation depend on external shuffle service to work well?
- Does Spark structured streaming support chained flatMapGroupsWithState by different key?
Related Questions in BUILD
- Build issue in my STM32-NUCLEO project using the Eclipse IDE
- Module not found when building flutter app for IOS
- Why am I getting this error ? error CS0103: The name 'EnhancedStackTrace' does not exist in the current context
- Gradle 8.7 cannot find installed JDK 22 in IntelliJ
- Build LLVM, Clang and Libfuzzer
- when I open a ktor project, error Cannot invoke "java.nio.file.Path.toString()" because the return value of "java.nio.file.Path.getFileName()" is null
- Cannot make Django run the frontend from Vite's build ("was blocked because of a disallowed MIME type (“text/html”)")
- Distorted CSS after Build process
- how to build nextjs app unable to build and deploy
- How to build custom mediapipe python model i.e. adding flow_limiter_calculator to face_landmark_front_cpu.binarypb
- Assets not showing after build process in Vite and React
- "Config.guess failed to determine the host type" when trying build binutils-2.7 with Cygwin
- The assembled Python application does not launch
- Why rebuild module does not recompile dependency module, but build module does in IntelliJ Idea?
- Gitlab pipeline stuck with nx cloud issue
Related Questions in SBT
- How to upload a native scala project to local repo by sbt like using "maven install"
- spark streaming and kafka integration dependency problem
- sbt publishLocal of a project with provided dependencies in build.sbt doesn't make these dependencies visible to projects using the project as library
- How to point sbt to use locally installed protoc in my scala project
- [email protected] Java 17 and sbt 1.9.8 and Scala 2.13 : report coverage is zero for all categories
- In chisel6.2.0, how to use hex file to init memory and test it?
- build.sbt error: value addRepositoryAuthentication is not a member of lmcoursier.CoursierConfiguration
- sbt not picking up dependencies with a crossProject
- How do I prevent WartRemover from interferring with the metals dap console?
- intellij: Cannot resolve symbol apache, jsoniter
- No lib of ivy cache jars created when I create package or install
- How do I update a multi project sbt build that depends on macro libraries to scala 3 in stages?
- Redownload snapshot dependency in sbt
- sbt-native-packager fails with dpkg-deb error
- Error initializing SparkContext caused by java.nio.channels.UnresolvedAddressException: null
Related Questions in APACHE-SPARK-2.0
- Spark SQL query fails with NullPointerException
- Parquet Size Difference
- Error importing Spark version 2.12.10 in Scala project Intellij IDE
- Unable to create spark pods in OCP with read write permission
- Spark-Hive Application: SASL Negotiation Failure with Kerberos on a Cluster
- Spark worker is killing the application
- Get day of week in Pyspark 3 using date format
- Maximum JDK version supported for apache spark 2.4.5.1
- How to free up Spark worker container memory after jobs finish
- Spark checkpointing data cleaning frequency
- Join 2 large size tables (50 Gb and 1 billion records)
- how to read CSV file with apache-spark spring boot
- Two big files join as one to many relationship in Java Spark
- How to configure pyspark to access AWS S3 containers?
- Writing parquet output with selected attributes from Bean
Related Questions in SBT-BUILDINFO
- Generate build info source file in Maven project (similar to SBT Buildinfo)
- What is buildInfoPackage in sbt-buildinfo?
- Conditional multi projects builds using sbt
- sbt-buildinfo plugin: How can I add the java version in effect at compile time to the generated BuildInfo.scala?
- sbt: generating shared sources in cross-platform project
- Set current project in sbt - spark build issue
- IntelliJ Idea can't resolve symbol that inside Scala managed sources (sbt-buildinfo)
- How to disable buildNumber increment in specific configurations?
- sbt-buildinfo generated object cannot be referenced
- How can I ask sbt-buildinfo to run only at (re)load time and not compile time?
- SBT: Access generated classes in Build.scala
- sbtbuildinfo - Building with multiple projects
- How to not watch a file for changes in Play Framework
- Can the runtime of a Heroku app know it's commit id?
- How to get Eclipse to resolve BuildInfo in an sbt / Scala/ BuildInfo project
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Popular Tags
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
So I was in the wrong direction. I was using the sbt script incorrectly for the most part. Looking at the spark dev tools I just needed an additional parameter like "build/abt compile"