WPF ContentPresenter Recreating Template

Question

I have strange behavior with the ContentPresenter in WPF. I use the ContentPresenter control in my user control to change some child controls based on a property change in my view model. The ContentPresenter definition looks like this:

        <ContentPresenter Grid.Row="2" Content="{Binding}">
            <ContentPresenter.Resources>
                <DataTemplate x:Key="Gray" d:DataType="{x:Type vm:ResultViewModel}">
                    <util:RestrictDesiredSize KeepHeight="True">
                        <Border Padding="5,5">
                            <controls:GrayAnalysis/>
                        </Border>
                    </util:RestrictDesiredSize>
                </DataTemplate>
                <DataTemplate x:Key="Color" d:DataType="{x:Type vm:ResultViewModel}">
                    <util:RestrictDesiredSize KeepHeight="True">
                        <Border Padding="5,5">
                            <controls:ColorAnalysis/>
                        </Border>
                    </util:RestrictDesiredSize>
                </DataTemplate>
            </ContentPresenter.Resources>
            <ContentPresenter.Style>
                <Style TargetType="{x:Type ContentPresenter}">
                    <Setter Property="ContentTemplate" Value="{StaticResource Gray}" />
                    <Style.Triggers>
                        <DataTrigger Value="True" Binding="{Binding Unit.IsColor.Value}">
                            <Setter Property="ContentTemplate" Value="{StaticResource Color}" />
                        </DataTrigger>
                        <DataTrigger Value="False" Binding="{Binding Unit.IsColor.Value}">
                            <Setter Property="ContentTemplate" Value="{StaticResource Gray}" />
                        </DataTrigger>                            
                    </Style.Triggers>
                </Style>
            </ContentPresenter.Style>
        </ContentPresenter>

So in general I have two templates and when the "Unit.IsColor.Value" property changes the corresponding data template should be applied. This seems to work in the first place.

When the correct data template was applied, another property change in the view model (for a different property) which is only bound to a control inside the applied template, the complete data template will be created again.

So instead of applying the data template only one time and then changing the control inside the data template, the ContentPresenter will recreate the complete data template for every property change.

The callstack looks like this:

    mscorlib.dll!System.Activator.CreateInstance(System.Type type, bool nonPublic)  Unknown
mscorlib.dll!System.RuntimeType.CreateInstanceImpl(System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, object[] args, System.Globalization.CultureInfo culture, object[] activationAttributes, ref System.Threading.StackCrawlMark stackMark)  Unknown
mscorlib.dll!System.Activator.CreateInstance(System.Type type, System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, object[] args, System.Globalization.CultureInfo culture, object[] activationAttributes) Unknown
mscorlib.dll!System.Activator.CreateInstance(System.Type type, object[] args)   Unknown
System.Xaml.dll!System.Xaml.Schema.SafeReflectionInvoker.CreateInstanceCritical(System.Type type, object[] arguments)   Unknown
System.Xaml.dll!System.Xaml.Schema.XamlTypeInvoker.CreateInstance(object[] arguments)   Unknown
System.Xaml.dll!MS.Internal.Xaml.Runtime.ClrObjectRuntime.CreateInstance(System.Xaml.XamlType xamlType, object[] args)  Unknown
System.Xaml.dll!System.Xaml.XamlObjectWriter.Logic_CreateAndAssignToParentStart(MS.Internal.Xaml.Context.ObjectWriterContext ctx)   Unknown
System.Xaml.dll!System.Xaml.XamlObjectWriter.WriteEndObject()   Unknown
PresentationFramework.dll!System.Windows.FrameworkTemplate.LoadTemplateXaml(System.Xaml.XamlReader templateReader, System.Xaml.XamlObjectWriter currentWriter)  Unknown
PresentationFramework.dll!System.Windows.FrameworkTemplate.LoadTemplateXaml(System.Xaml.XamlObjectWriter objectWriter)  Unknown
PresentationFramework.dll!System.Windows.FrameworkTemplate.LoadOptimizedTemplateContent(System.Windows.DependencyObject container, System.Windows.Markup.IComponentConnector componentConnector, System.Windows.Markup.IStyleConnector styleConnector, System.Collections.Generic.List<System.Windows.DependencyObject> affectedChildren, System.Windows.UncommonField<System.Collections.Hashtable> templatedNonFeChildrenField)   Unknown
PresentationFramework.dll!System.Windows.FrameworkTemplate.LoadContent(System.Windows.DependencyObject container, System.Collections.Generic.List<System.Windows.DependencyObject> affectedChildren)    Unknown
PresentationFramework.dll!System.Windows.StyleHelper.ApplyTemplateContent(System.Windows.UncommonField<System.Collections.Specialized.HybridDictionary[]> dataField, System.Windows.DependencyObject container, System.Windows.FrameworkElementFactory templateRoot, int lastChildIndex, System.Collections.Specialized.HybridDictionary childIndexFromChildID, System.Windows.FrameworkTemplate frameworkTemplate) Unknown
PresentationFramework.dll!System.Windows.FrameworkTemplate.ApplyTemplateContent(System.Windows.UncommonField<System.Collections.Specialized.HybridDictionary[]> templateDataField, System.Windows.FrameworkElement container)   Unknown
PresentationFramework.dll!System.Windows.FrameworkElement.ApplyTemplate()   Unknown
PresentationFramework.dll!System.Windows.FrameworkElement.MeasureCore(System.Windows.Size availableSize)    Unknown
PresentationCore.dll!System.Windows.UIElement.Measure(System.Windows.Size availableSize)    Unknown
PresentationCore.dll!System.Windows.ContextLayoutManager.UpdateLayout() Unknown
PresentationCore.dll!System.Windows.ContextLayoutManager.UpdateLayoutCallback(object arg)   Unknown
PresentationCore.dll!System.Windows.Media.MediaContext.FireInvokeOnRenderCallbacks()    Unknown

It seems like the MeasureCore is tiggering the ApplyTemplate and so forth...

So the question is, how can I avoid that the template is recreated for every property call?