How to calculate the entropy of a coin flip

Question

I would like to know ...

In the repeated coin flip, ¿how do I calculate the entropy of the random variable X that represents the number of flips to do until get "head" for first time?

Answer 1

The variable X can take any number from 1 through infinity. The probabilities are:

p(X = i) = (1/2)^i

The entropy is:

H = - Sum {i from 1 to infinity} ( p(X = i) * log2(p(X = i)) )
  = - Sum {i from 1 to infinity} ( 1/2^i * log2(1/2^i) )
  = - Sum {i from 1 to infinity} ( 1/2^i * i * log2(1/2) )
  =   Sum {i from 1 to infinity} ( 1/2^i * i )

Solving this yields:

H = 2 bit