Problem statement: Design a module that can be consumed by the customers to push data (metadata of varying entity. e.g. Product, Car and similar POJO) from their boundary to our boundary. The data is from the hot path and it may not be fetched again. This requirement is very much like high performance log agent which push log from the service to external storage except we are pushing metadata of different entities in this case.
Constraints : It should have limited impact on customer's hot path. External resources like Queue shouldn't be into the Customer's boundary.
Why Agent based module? : The idea is that agent will persist the data locally and can re-try sending the data in case of failures. I may use a Queue in our boundary but the Customer Service may fail to push the data to the Queue resulting in loss of data.
Here's how the design would look like.
- Service A is in the Customer's boundary. Agent can reside in the Service A of the Customer from where it would push the data to our Queue.
- The Queue is an optional resource added in our boundary. We have a single Service that consumes the message being pushed from the agent and we may consume the message directly bypassing the Queue.
- The agent persists the data locally in the Customer's VM and can push the data over a period of time.
Question : Can we use Azure Monitor Agent for the purpose of pushing metadata through a rest API? Is there any existing agent that can be re-used? Basically I'm looking for a way to store metadata in a file system and then push it through rest API.