For a distribution of N values, how can I efficiently upper-bound the largest divergence between all non-negative distributions over the same random field? For example, for all distributions of a random variable that takes values in ([1,2,3,4]), i.e., N = 4, and the probability of a = 1 or a = 2 or a = 3 or a = 4 is always nonzero (but can be very small, e.g., 1e-1000).
Is there a known bound (other than infinity)? Say, given the number N, the divergence between the uniform distribution [1/4 1/4 1/4 1/4] and "delta" [1e-10 1e-10 1e-10 1/(1+3e-10)] over N is the largest?...
Thanks all in advance, A.