creating monitor dashboard using cloud run service logs

Question

I need assistance in validating the process of creating dashboard in GCP. I have one service running on cloud run. When an event triggers where we log the object in logger statement with message. In this message we have one field with value. We wanted to show the sum of the value of that field each time it get logged in gcp.

Ex:

{"testRecord":{"id":"815582a0f344179be1d","totalAmount":2.00,"totalvalue":0.50}}

I have created log based metrics. These Log-based metrics count the log entries that match a given filter.

Using the regex expression which extracts a value from the field that must contain exactly one regex group (), I can extract the value and then in monitoring dashboard, I can select the metric type as distribution which collects numeric data from log entries.

The issue with this is, logs are only retained till 30 days from creation and log metrcis does not fetches logs before creation, in this case the data will not be accurate.

Can someone please suggest me if we have any alternative way to create monitor dashboard.

Answer 1

To achieve the aggregation and visualization of the TotalAmount field value from the log messages in Google Cloud Platform (GCP), you can use Cloud Monitoring and create a custom metric to aggregate and display the sum of the TotalAmount field over time as per below steps :

• In the GCP Console navigate to the Logging section and select the Log based metrics. These Log-based metrics count the log entries that match a given filter. This is Project scoped and only applies to logs generated in this project.

  Click on Create metric , select the metric type as distribution which collects numeric data from log entries matching a given filter. Define the log based metric in the filter section accordingly for which you are looking for. As you are looking for TotalAmount, select the filter as jsonPayload.message and give the field as mentioned in this official doc. Fill the label section accordingly and give the Message value what you are looking for, a regular expression which extracts a value from the field that must contain exactly one regex group (). Click on on create and this will Create a log based metric .

Below is the screenshot for reference :

• After creating log bases metric as shown in above screenshot, click on the three dots at right corner and select view in the metrics explorer. You can see as below :

• You can add filters and aggregation as per your requirement. Data will be shown as above and then click on the save chart then give the chart title and dashboard name, this will create the dashboard for the metric which you have created.

You can see the dashboard output as below :

Refer to this Google Community solution for information.