Send logs from AWS to Elasticcloud

Question

I am using Elasticcloud (hosted elasticsearch) to index my app data. Now I want to start streaming logs from my AWS lambda functions to my Elasticcloud account. I have googled and I can see that there are couple of ways to do this:

1. Functionbeat
2. Cloudwatch-> Elasticsearch subscription filter
3. Cloudwatch-> Lambda subscription filter

My questions are

• which is the most cost efficient and performant way to stream logs from AWS cloudwatch to elasticcloud
• For functionbeat is it necessary to first send logs to a S3 bucket? (I am referring to this https://www.elastic.co/guide/en/beats/functionbeat/current/configuration-functionbeat-options.html)

Answer 1

First question:

Since Functionbeat is deployed to Lambda in case of AWS, no.1 and no.3 cost the same. No.1 is faster to deploy because you need to create Lambda by yourself in no.3.

As for performance, of course it depends on the implementation, I guess there is no big difference between two methods unless millisecond latency has impact to you.

If you are using Elastic Cloud you can't use no.2, which works with Amazon Elasticsearch Service. These two are completely different services. (see this page, I know it's a bit confusing!)

---

Second question:

No, you don't have to. Functionbeat directly gets logs from CloudWatch. S3 bucket is used to store Function beat module itself before being deployed to Lambda.