I have been struggling to redact/obfuscate logs using tranform processor.
This is Splunk Otel distribution implementation on EKS cluster.
The logs looks like this:
2024-03-11T21:04:41.411025006Z stdout F {"time": "2024-03-11T21:04:41+00:00", "upstream_namespace":"system-monitoring", "remote_user": "sample-user"}
Processor Configuration( tested individually, just clubbed here for ease):
processors:
attributes/upsert:
actions:
- key: upstream_namespace
action: upsert
value: "REDACTED_NS"
transform:
log_statements:
- context: log
statements:
- replace_all_patterns(attributes,"value","upstream_namespace", "REDACTED_NS")
- replace_all_patterns(attributes,"key","upstream_namespace", "REDACTED_NS")
- replace_match(attributes["upstream_namespace"], "*" , "REDACTED_NS")
- replace_match(attributes["upstream_namespace"], "system-monitoring" , "REDACTED_NS")
- delete_key(attributes,"upstream_namespace")
- delete_key(resource.attributes,"upstream_namespace")
- replace_all_patterns(attributes["upstream_namespace"],"value","upstream_namespace", "REDACTED_NS")
- replace_all_patterns(attributes["upstream_namespace"],"value","system-monitoring", "REDACTED_NS")`
The attribute/upsert however adds REDACTED_NS value along with the original.
upstream_namespace: REDACTED_NS
system-monitoring
Any suggestions to achieve this logs transformation?