Unity - Content Size Fitter not Working Correctly with Multiple Levels

Question

I have a case where I'm trying to use 3 Content Size Fitters at 3 different levels but only the lowest one seems to work correctly.

Here's the layout.

ScrollRect
|-Content Panel - Vertical Group Layout, Child Force Expand Width and Height checked
   |- Panel2 - Content Size Fitter, Vertical Fit = Preferred Size
        |- Panel3 - Content Size Fitter, Vertical Fit = Preferred Size
             |- Text Mesh Pro (multiline) - Content Size Fitter, Vertical Fit = Preferred Size

Since I couldn't get this to work I created a simple script and put it on Panel2 and Panel3.

[ExecuteInEditMode]
public class PanelLayoutUpdater : MonoBehaviour
{
    [SerializeField]
    private GameObject childObject;

    private RectTransform panelRectTransform;
    private RectTransform childRectTransform;

    private float width;

    private void Update()
    {
        AdjustPanelHeight();
    }    

    void Start()
    {
        panelRectTransform = GetComponent<RectTransform>();
        width = panelRectTransform.sizeDelta.x;
        childRectTransform = childObject.GetComponent<RectTransform>();
    }

    void AdjustPanelHeight()
    {
        if (childRectTransform != null)
        {
            Vector2 childSize = childRectTransform.sizeDelta;
            panelRectTransform.sizeDelta = new Vector2(width, childSize.y + 10f);
        }
    }
}

The issue with this approach is the time it takes to size the Content. After running the test, I expect the scroll rect to be at the bottom but it's a few rows above the bottom, every time.

    private void Test()
    {
        var msg = @"New Text this is going to be a longer message to see if the text field auto sizes to allow for more text to be shown. If this works, it should resize the parent panel which should in-turn resize it's parent panel.

                The content size fitter works here

                Does this still work";

        for (int i = 0; i < 10; i++)
        {
            var msgObj = Instantiate(rightMessageTemplate, Content.transform);
            msgObj.GetComponent<ChatMessage>().Text.text = $"Right: {msg}"; 

            msgObj = Instantiate(leftMessageTemplate, Content.transform);
            msgObj.GetComponent<ChatMessage>().Text.text = $"Left: {msg}";
            yield return new WaitForSeconds(.1f);
        }
        
        LayoutRebuilder.ForceRebuildLayoutImmediate(transform as RectTransform);      
        scrollRect.verticalNormalizedPosition = 0f;        

        StartCoroutine(LastUpdate());
    }

I run into the same issue adding just one new item with a much longer text, maybe 50 lines.