Exporting JSON into single file from loop function

697 views Asked by At

I wrote some code which hits one public API and saves the JSON output in a file. But the data is storing line by line into the file instead of a single JSON format.

For eg.

Current Output:

{"ip":"1.1.1.1", "Country":"US"}
{"ip":"8.8.8.8", "Country":"IN"}

Desired Output:

[
{"ip":"1.1.1.1", "Country":"US"},
{"ip":"8.8.8.8", "Country":"IN"}
]

I know this should be pretty simple and i am missing out something.

My Current Code is:

To read IP from file and hit the API one by one on each IP.

func readIPfromFile(filename string, outFile string, timeout int) {
    data := jsonIn{}
    //open input file
    jsonFile, err := os.Open(filename) //open input file
    ...

    ...
    jsonData := bufio.NewScanner(jsonFile)

    for jsonData.Scan() {

        // marshal json data & check for logs
        if err := json.Unmarshal(jsonData.Bytes(), &data); err != nil {
            log.Fatal(err)
        }
        //save to file
        url := fmt.Sprintf("http://ipinfo.io/%s", data.Host)
        GetGeoIP(url, outFile, timeout)

    }
}

To make HTTP Request with custom request header and call write to file function.

func GetGeoIP(url string, outFile string, timeout int) {
    geoClient := http.Client{
        Timeout: time.Second * time.Duration(timeout), // Timeout after 5 seconds
    }

    req, err := http.NewRequest(http.MethodGet, url, nil)
    if err != nil {
        log.Fatal(err)
    }

    req.Header.Set("accept", "application/json")

    res, getErr := geoClient.Do(req)
    if getErr != nil {
        log.Fatal(getErr)
    }

    if res.Body != nil {
        defer res.Body.Close()
    }

    body, readErr := ioutil.ReadAll(res.Body)
    if readErr != nil {
        log.Fatal(readErr)
    }

    jsonout := jsonOut{}
    jsonErr := json.Unmarshal(body, &jsonout)
    if jsonErr != nil {
        log.Fatal(jsonErr)
    }
    file, _ := json.Marshal(jsonout)
    write2file(outFile, file)
}

To Write data to file:

func write2file(outFile string, file []byte) {
    f, err := os.OpenFile(outFile, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0600)
    if err != nil {
        log.Fatal(err)
    }

    defer f.Close()
    if _, err = f.WriteString(string(file)); err != nil {
        log.Fatal(err)
    }
    if _, err = f.WriteString("\n"); err != nil {
        log.Fatal(err)
    }

I know, i can edit f.WriteString("\n"); to f.WriteString(","); to add comma but still adding [] in the file is challenging for me.

1

There are 1 answers

0
Minh On

First, please do not invent a new way of json marshaling, just use golang built-in encoding/json or other library on github.

Second, if you want to create a json string that represents an array of object, you need to create the array of objects in golang and marshal it into string (or more precisely, into array of bytes)

I create a simple as below, but please DIY if possible.

https://go.dev/play/p/RR_ok-fUTb_4