Merge url with parameters into 1 in Splunk

Question

I am creating a dashboard for our service. And I want to create metrics for url requests. Lets say have a similar url like this one:

/api/v1/users/{userId}/settings

And I have following query in Splunk

url=*/api/v1/users/*/settings 
| stats avg(timeTaken) as avg_latency, p99(timeTaken) as "p99(ms)", perc75(timeTaken) as "p75(ms)", count  as total_requests, count(eval(responseStatus=500)) as failed_requests by url 
| eval "success_rate"=round((total_requests - failed_requests)/total_requests*100,2) 
| eval avg = round(avg) 
| sort success_rate

All I want is to have a table with one common url showing all the metrics. But instead, I get a table with a list of all urls with different parameters.

Answer 1

You want to create a field which is the URL minus the UserId part, And therefore the stats will be grouped by which url is called.

You can do this by using split(url,"/") to make a mv field of the url, and take out the UserId by one of two ways depending on the URLs.

Mvfilter: Eg: mvfilter(eval(x!=userId))

Or created a new mvfield with the userId removed by it's index in the mvfield using this: Add/Edit/Delete mvfield

Instead of removing you could also choose to replace the UserId with "{userId}", so long as you do the same for all Urls.

And then you can rejoin the url using mvjoin(url,"/")

I hope I understood your question correctly and this helps you!

Answer 2

You could try doing a replace() on your URL field with eval before calling stats:

| eval url=replace(url,"\/\d+\/settings","/settings")

If it turns out the userid is important to hold onto, pull it into its own field prior to running replace():

| rex field=url "\/(?<userid>\d+)\/settings"

expansion for comment

For multiple possible endings of your URL, try something like this:

index=ndx sourcetype=srctp URL IN("*/api/v1/users/*/settings","*/api/v1/users/*/logout","*/api/v1/users/*/profile")
| rex field=url "\/(?<url_type>\w+)$"
| eval url=replace(url,"\/\d+\/\w+$","")
| stats avg(timeTaken) as avg_latency, p99(timeTaken) as "p99(ms)", perc75(timeTaken) as "p75(ms)", count  as total_requests, count(eval(responseStatus=500)) as failed_requests by url type
| eval "success_rate"=round((total_requests - failed_requests)/total_requests*100,2) 
| eval avg = round(avg) 
| sort success_rate

This will extract the "type" (logout, profile, settings) into a new field, then cleanup the URL by removing everything from userid to the end