Writing per line new CSV File (JAVA)

Question

I have the following code:

public static void main(String[] args) throws IOException {
        //File being read:
        String fileName = "src/data/Belgium.csv";


    String[] nextLine;

    try (CSVReader reader = new CSVReader(new FileReader(fileName), ',', '"', 1)) {


        while ((nextLine = reader.readNext()) != null) {

            for (String line : nextLine) {
                //NewFile
                //When 2nd parameter - ture, it gets so big, that excel can't handle it anymore...
                FileWriter writer = new FileWriter("src/dataNew/BelgiumNew1.csv", true);

                line = line.replaceAll("T", " ");
                line = line.replaceAll("Z", "");
                line = line.replaceAll("ActualGenerationPerUnit.mean", "");
                line = line.replaceAll("Plantname:", "");
                //Escaping curly braces is a must!
                line = line.replaceAll("\\{", "");
                line = line.replaceAll("\\}", "");

                writer.append(line);
                writer.flush();
                writer.close();

                System.out.println(line);
            }
        }System.out.println("Successfully written");
    }
}

The output of the code in my console, using System.out.println(line) gives me the correct output. However, when I open the CSV file, it seems like it is written reversed. Excel first complains about the amount of rows. However, only the last row of my original dataset shows. The dataset (which is preprocessed in an inefficient way), contains more than 1000 rows. Therefore, I can not simply append every single entry.

Is there a better way of doing this?

Tips and tricks are very welcome. Furtermore, I have tried several writers: - CSVwrite - BufferedWriter - FileWriter

Also checked other issues on Stackoverflow... Can't seem to make it work. Thank you!

UPDATE:

Question is answered! Final code:

 public static void main(String[] args) throws IOException {
    //File being read:
    String fileName = "src/data/Belgium.csv";

    //When 2nd parameter - ture, it gets so big, that excel can't handle it anymore...
      FileWriter writer = new FileWriter("src/dataNew/BelgiumNew5.csv", true);


    String[] nextLine;

    try (CSVReader reader = new CSVReader(new FileReader(fileName), ',', '"', 1)) {

        while ((nextLine = reader.readNext()) != null) {

            for (String line : nextLine) {

                line = line.replaceAll("T", " ");
                line = line.replaceAll("Z", "");
                line = line.replaceAll("ActualGenerationPerUnit.mean", "");
                line = line.replaceAll("Plantname:", "");
                //Escaping curly braces is a must!
                line = line.replaceAll("\\{", "");
                line = line.replaceAll("\\}", "");

                writer.append(line);
                writer.append(System.lineSeparator());
                System.out.println(line);


            }
        }System.out.println("Successfully written");
        }catch(Exception e){
        e.printStackTrace();
    }finally {
        if (writer != null){
            writer.flush();
            writer.close();
        }
    }
}

Answer 1

However, when I open the CSV file, it seems like it is written reversed. Excel first complains about the amount of rows. However, only the last row of my original dataset shows.

I think that it is probably caused because a new line character between CSV rows is missing.

Actually you don't write a new line character when you write a row in the file. You could write it : writer.append(System.lineSeparator()) after each read line.

As side notes :

1) Why not moving it before the loop (otherwise it is less efficient) :

FileWriter writer = new FileWriter("src/dataNew/BelgiumNew1.csv", true);

2) You should not flush and close the file at each read line as it is less efficient :

writer.append(line);
writer.flush();
writer.close();
System.out.println(line);

It should be enough :

writer.append(line);
System.out.println(line);

Keep that :

writer.flush();
writer.close();

in a finally statement such as :

FileWriter writer = new FileWriter("src/dataNew/BelgiumNew1.csv", true);

try{
  // all your operations to handle the file
}

catch(Exception e){
  // your exception handling
}

finally{
   if (writer!=null){
      writer.flush();
      writer.close();
   }
}

---

EDIT to answer to the comment :

If you have the impression that the output file contains multiple set of records, it is probably related to the append mode of the FileWriter that you used.

Replace that :

FileWriter writer = new FileWriter("src/dataNew/BelgiumNew1.csv", true);

by this to not use this mode :

FileWriter writer = new FileWriter("src/dataNew/BelgiumNew1.csv", false);

Answer 2

I see you are using opencsv to read your csv file. In addition to the correct answer from davidxxx you could simplyfy your code if you use the CSVWriter from opencsv to write to file. Below is an example:

import com.opencsv.CSVReader;
import com.opencsv.CSVWriter;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.List;

public class Example1 {

    public static void main(String args[]) throws IOException {                              
         String fileName = "src/data/Belgium.csv";
         CSVReader reader = new CSVReader(new FileReader(fileName), ',','"',1);
         List<String[]> lines = reader.readAll();         
         CSVWriter writer = new CSVWriter(new FileWriter("src/data/BelgiumNew1.csv",false), ',');
         for(String[] row : lines){
             for(int i = 0; i< row.length; i++){
                row[i] = row[i].replaceAll("T", " ")
                         .replaceAll("Z", "z")
                         .replaceAll("ActualGenerationPerUnit.mean", "")
                         .replaceAll("Plantname:", "")
                         .replaceAll("\\{", "")
                         .replaceAll("\\}", "");                 
             }
             writer.writeNext(row);
         }
         writer.flush();
         writer.close();
    }
}