Data race issues while reading data from file and sending it simultaneously

303 views Asked by At

I am trying to read data from a file and send it immediately the chunk read without waiting the other goroutine to complete file read. I have two function

func ReadFile(stream chan []byte, stop chan bool) {
    file.Lock()
    defer file.Unlock()
    dir, _ := os.Getwd()
    file, _ := os.Open(dir + "/someFile.json")
    chunk := make([]byte, 512*4)
    for {
        bytesRead, err := file.Read(chunk)
        if err == io.EOF {
            break
        }
        if err != nil {
            panic(err)
            break
        }
        stream <- chunk[:bytesRead]
    }

    stop <- true
}

First one is responsible for reading the file and sending data chunks to the "stream" channel received from the other function

func SteamFile() {
    stream := make(chan []byte)
    stop := make(chan bool)

    go ReadFile(stream, stop)

L:
    for {
        select {
        case data := <-stream:
            //want to send data chunk by socket here
            fmt.Println(data)
        case <-stop:
            break L
        }
    }
}

Second function creates 2 channel, sends them to the first function and starts to listen to the channels. The problem is that sometimes data := <-stream is missing. I do not receive for example the first part of the file but receive all the others. When I run the program with -race flag it tells that there is a data race and two goroutines write to and read from the channel simultaneously. To tell the truth I thought that's the normal way to use channels but now I see that it brings to even more issues. Can someone please help me to solve this issue?

1

There are 1 answers

3
Peter On BEST ANSWER

When I run the program with -race flag it tells that there is a data race and two goroutines write to and read from the channel simultaneously. To tell the truth I thought that's the normal way do use channels.

It is, and you are almost certainly misunderstanding the output of the race detector.

You're sharing the slice in ReadFile (and therefore it's underlying array), so the data is overwritten in ReadFile while it's being read in SteamFile [sic]. While this shouldn't trigger the race detector it is definitely a bug. Create a new slice for each Read call:

  func ReadFile(stream chan []byte, stop chan bool) {
      file.Lock()
      defer file.Unlock()
      dir, _ := os.Getwd()
      file, _ := os.Open(dir + "/someFile.json")
-     chunk := make([]byte, 512*4)
      for {
+         chunk := make([]byte, 512*4)
          bytesRead, err := file.Read(chunk)