Jest client shutdown after the first execute operation

661 views Asked by At

I created an AWS Lambda package (Java) with a function that reads some files from Amazon S3 and pushes the data to AWS ElasticSearch Service. Since I'm using AWS Elastic Search, I can't use the Transport client, in which case I'm working with the Jest Client to push via REST. The issue is with the Jest client.

Here's my Jest client instance:

public JestClient getClient() throws InterruptedException{
    final Supplier<LocalDateTime> clock = () -> LocalDateTime.now(ZoneOffset.UTC);

    DefaultAWSCredentialsProviderChain awsCredentialsProvider = new DefaultAWSCredentialsProviderChain();

    final AWSSigner awsSigner = new AWSSigner(awsCredentialsProvider, REGION, SERVICE, clock);

    JestClientFactory factory = new JestClientFactory() {
        @Override
        protected HttpClientBuilder configureHttpClient(HttpClientBuilder builder) {
            builder.addInterceptorLast(new AWSSigningRequestInterceptor(awsSigner));
            return builder;
        }

        @Override
        protected HttpAsyncClientBuilder configureHttpClient(HttpAsyncClientBuilder builder) {
            builder.addInterceptorLast(new AWSSigningRequestInterceptor(awsSigner));
            return builder;
        }
    };

    factory.setHttpClientConfig(
            new HttpClientConfig.Builder(URL)
                    .discoveryEnabled(true)
                    .multiThreaded(true).build());

    JestClient jestClient = factory.getObject();
    return jestClient;
}

Since the AWS Elasticsearch domain is protected by an IAM access policy, I sign the requests for them to be authorized by AWS (example here). I use POJOs to index documents.

The problem I face is that I am not able to execute more than one action with the jest client instance. For example, if I created the index first :

client.execute(new CreateIndex.Builder(indexName).build()); 

and later on, I wanted to, for example do some bulk indexing:

for (Object object : listOfObjects) {
    bulkIndexBuilder.addAction(new Index.Builder(object ).
                      index(INDEX_NAME).type(DOC_TYPE).build());
        }
client.execute(bulkIndexBuilder.build());

only the first action will be executed and the second will fail. Why is that? Is it possible to execute more than one action?

Morover, using the provided code, I'm not able to execute more than 20 Bulk operations when I want to index the document. Basically, around 20 is fine, but anything more than that, the client.execute(bulkIndexBuilder.build()); just does not execute and the client shuts down.

Any help or suggestion would be appriciated.

UPDATE: It seems that AWS ElasticSearch does not allow connecting to individual nodes. Simply turning off node discovery in the Jest client .discoveryEnabled(false) solved all the problems. This answer helped.

0

There are 0 answers