I want to use ND4J purely as linear algebra library and would like to create an INDArray directly from either a csv file or a spark dataframe without having to create an intermediate Java collection on-heap. I tried to find a solution everywhere but couldn’t find anything.
Create INDArray (ND4J) from csv or spark data frame
58 views Asked by Lukas Tycho At
1
There are 1 answers
Related Questions in JAVA
- I need the BIRT.war that is compatible with Java 17 and Tomcat 10
- Creating global Class holder
- No method found for class java.lang.String in Kafka
- Issue edit a jtable with a pictures
- getting error when trying to launch kotlin jar file that use supabase "java.lang.NoClassDefFoundError"
- Does the && (logical AND) operator have a higher precedence than || (logical OR) operator in Java?
- Mixed color rendering in a JTable
- HTTPS configuration in Spring Boot, server returning timeout
- How to use Layout to create textfields which dont increase in size?
- Function for making the code wait in javafx
- How to create beans of the same class for multiple template parameters in Spring
- How could you print a specific String from an array with the values of an array from a double array on the same line, using iteration to print all?
- org.telegram.telegrambots.meta.exceptions.TelegramApiException: Bot token and username can't be empty
- Accessing Secret Variables in Classic Pipelines through Java app in Azure DevOps
- Postgres && statement Error in Mybatis Mapper?
Related Questions in ND4J
- "org.nd4j.linalg.factory.Nd4j.randomFactory" is null in Deeplearning4j
- Deeplearning4j: System Freezes During Multi-threaded Inference despite Separate Model Instances
- High Memory Usage with Deeplearning4j during Model Inference
- VGG16 pre-trained model is not giving accurate Image Similarity in scala
- DL4J/ND4J Model Fitting Error: Rank of Input Data Drops to 0 After Slicing INDArray for Training Subset
- Memory Leak in Nd4j INDArray
- Nd4j and flink memory leak
- Create INDArray (ND4J) from csv or spark data frame
- why can't Maven find the nd4j .jar file after adding the dependency to pom.xml?
- ND4J NoAvailableBackendException despite following all documentation steps
- Runtime Error From ND4J When Executing SVD
- Problem Adding A Column Vector To A Matrix Using ND4J
- Cannot resolve symbol 'linspace' in ND4j
- BooleanIndexing.replaceWhere Not Working Based On Type
- Calculating the principal axis/eigenvalues and -vectors of large dataset in Java
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Popular Tags
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Nd4j maintainer here. If you want to read just use a map over a spark data frame, have each step create a row and then combine the final result using Nd4j.concat(..).
If you can, consider converting to arrow first and then using the nd4j-arrow converter.It uses the tensor abstraction which didn't seem to get as much support as I hoped. You can find that here: https://github.com/deeplearning4j/deeplearning4j/blob/master/nd4j/nd4j-serde/nd4j-arrow/src/main/java/org/nd4j/arrow/ArrowSerde.java#L42
Feel free to file an issue over on the dl4j repository and I can take a look at other recommendations for you as well.