In state pattern, what's the best way to instantiate/concrete next state object in C# since I can't release last state object

Question

I'm working on a wpf project which need to apply state pattern. And I've read a great article Game programming pattern - state.

And I know there are two approaches to instantiate/concrete the next state object.

One for dynamically instantiate it and another one is to hold the reference in state machine and at compile time, eg. in constructor, concrete them.

And since in my working project, I have many different of derived states and difference combinations of them. So I choose to dynamically instantiate the next state object.

But I notice one problem that since I can't release the state by my own, so every time I create next state, there will be an un-reference state object left in the memory to wait for GC to collect at uncertain time.

And that makes me wonder maybe I design the state pattern in the wrong way.

Following is the pseudo code of my state pattern.

public enum MyTrigger
{
    Start,
    Pause,
    Stop,
    Resume,
}

public interface IState
{
    void Handle(StateMachine context, MyTrigger trigger);
}

public class BeginState : IState
{
    public void Handle(StateMachine context, MyTrigger trigger)
    {
        if (trigger == MyTrigger.Start)
            context.SetState(new RunningState());
    }
}

public class RunningState : IState
{
    public void Handle(StateMachine context, MyTrigger trigger)
    {
        if (trigger == MyTrigger.Pause)
            context.SetState(new PauseState());
    }
}

public class PauseState : IState
{
    public void Handle(StateMachine context, MyTrigger trigger)
    {
        if (trigger == MyTrigger.Stop)
            context.SetState(new BeginState());
        else if (trigger == MyTrigger.Resume)
            context.SetState(new RunningState());
    }
}

public class StateMachine
{
    private IState mCurrentState;

    public StateMachine(IState beginState)
    {
        mCurrentState = beginState;
    }

    public void SetState(IState state)
    {
        mCurrentState = state;
    }

    public void Handle(MyTrigger trigger)
    {
        mCurrentState.Handle(this,trigger);
    }
}