Update individual instance of Azure Cloud Service

Question

I have a cloud service hosted out in Azure. Due to the amount of data it uses, we have an hourly process that tells the service to cache data into memory. To update the cache, we use an exposed method called "RefreshData" that we invoke on a scheduled task on a local server.

Recently I updated the cloud service to run on 2 instances (meaning it was running on 1 VM, its now running on 2).

What Im noticing now is that our data is no longer automatically refreshing to the most current data. What seems to be happening here is:

1. A scheduled task to run a command executable is triggered at the top of the hour on a local server
2. The exe calls out to http://myCloudService.cloudapp.net/service.svc, gets a response, and kicks off the RefreshData method
3. The instance that received the refresh request is "random", meaning only 1 of the 2 instances is told to refresh the data. Therefore, sometimes a user using our service gets most current data, sometimes it can be up to a few hours stale.

So if I have a cloud service that responds to an IP, but there are 2 instances running "behind it" with individual IP's, how can I invoke this request and ensure that both are getting updated as needed?

PS - I'm aware that Azure has that "load balancer" tool now available, if needed Im perfectly comfortable using one instance as my primary and have the second instance work as a failover, if thats a plausable option here?

Answer 1

You have a few options:

1. Why are your cloud services themselves not responsible for refreshing their own data on a schedule? Why do they have to be told to do this from external? Using a scheduled task or timer inside the role instances themselves would be the easiest and most reliable method.
2. Use an InstanceInputEndpoint so that each instance gets a unique port number. Then your scheduled task would make two calls, one to http://myCloudService.cloudapp.net:81/service.svc and one to http://myCloudService.cloudapp.net:82/service.svc.
3. Setup an instance level public IP address (http://azure.microsoft.com/blog/2014/10/22/instance-level-public-ip-address/). Your scheduled task would make two calls, one to http://{instanceIP1}/service.svc, and one to http://{instanceIP2}/service.svc.
4. Use the same configuration you are using now, where a random instance behind the load balancer gets the refresh command, and then internally that instance would communicate to the other instance(s) using InternalEndpoints.

Answer 2

Your current solution is overengineered big time. You have an external service just to trigger a periodic event inside you Azure service. What if your ISP fails and you cannot connect from one to another for a while?

A much better approach would be for each instance to send an HTTP request to itself - effectively http://localhost/service.svc That would eliminate the dependency on an external service and also solve the problem with going through the load balancer and reaching each instance.

I assume you have a web role. Depending on what your RoleEntryPoint.Run() does you can either start separate thread from there or just run an infinite loop with Sleep() inside and send those HTTP requests from there.