Elastic Search returning null after some time

55 views Asked by At

Elastic Search starts returning null after some time.

I have written a go client which will send a request to send a request to my self hosted elastic-search cluster. After debugging for a while, came to know might be I would be making an insecure call to self hosted elastic-search cluster so I modified my go client to make a secure connection with elastic-search every-time I make an API request.

The go client looks like this:

func getResponseFromElastic(payload string) Response {
    tr := &http.Transport{
        TLSClientConfig: getTLSConfig(ConfigStruct.ElasticStruct.CaCert),
        Proxy:           nil,
    }

    data := strings.NewReader(payload)
    client := &http.Client{Transport: tr}

    elasticUrl := ConfigStruct.ElasticStruct.ElasticUrl
    elasticIndex := ConfigStruct.ElasticStruct.Index
    httpUrl := elasticUrl + elasticIndex + "/_search"
    req, err := http.NewRequest("GET", httpUrl, data)

    var response Response = Response{}

    if err != nil {
        ErrorLogger.Println(err)
        return response
    }
    req.Header.Set("Content-Type", "application/json")
    req.SetBasicAuth(ConfigStruct.ElasticStruct.ElasticUsername, ConfigStruct.ElasticStruct.ElasticPassword)
    resp, err := client.Do(req)

    if err != nil {
        ErrorLogger.Println(err)
        return response
    }
    defer func() {
        err := resp.Body.Close()
        if err != nil {
            ErrorLogger.Println(err)
        }
    }()
    bodyText, err := io.ReadAll(resp.Body)
    if err != nil {
        ErrorLogger.Println(err)
        return response
    }
}

getgetTLSConfig function looks like this:

func getTLSConfig(caCertFile string) *tls.Config {
    var caCert []byte
    var err error
    var caCertPool *x509.CertPool
    caCert, err = ioutil.ReadFile(caCertFile)
    if err != nil {
        ErrorLogger.Fatal("Error opening cert file", caCertFile, ", error ", err)
    }
    caCertPool = x509.NewCertPool()
    caCertPool.AppendCertsFromPEM(caCert)

    return &tls.Config{
        InsecureSkipVerify: false,
        RootCAs:            caCertPool,
    }
}

My ConfigStruct.ElasticStruct.ElasticUrl looks like

"https://self-hostedes001:9200/",

Things I have tried,

  • Increasing the JVM heap size of the elastic search nodes, so in case if it clocks for higher memory usage, it should not block the user.
  • Adding ca certs to my API requests so that es is able to identify if the user is a identified user or not.
  • So once Elastic Search starts returning null response, I have a temporary fix for that, I have to restart the elastic-search cluster. Once I restart, Elastic search returns response normally but it again return null after some time.
0

There are 0 answers