My experiment is based on an online ab test system. A typical situation is that, I generate some candidates with some algorithm (CEM, GP etc) and push them to different experiment buckets (around 10). After one day, I will get a daily ab test report for previous day. This will be my evaluated result for these candidates, which is reward for next iteration. But the tricky thing is that, even for the same candidates, the result will not be same in different two days because the environment itself is not stable.
So my question is that: is there any black-box optimizer suitable for my situation, in which the environment is not so stationary? or is there any way to take the environment into consideration? Do I need to try neural network in which I could add more feature describing the environment? (then it will not be a black-box problem, but the cost is affordable to make the change)