Flutter layout with minimum height elements which falls back to scrolling upon overflow

Question

I would like to create a layout in Flutter that follows a few simple rules, illustrated by the image below.

1. If all of the arbitrary views in my layout are vertically shorter than the parent widget's available space, they lay out based on their intrinsic size.
2. If the vertical space is confined such that not all of the views fit within the given space, certain elements have a dynamic height and can shrink to some arbitrary minimum height (e.g. spacers between views, or the color in my example).
3. If the vertical space is confined such that not all of the views can fit within the given space, even when the elements with dynamic height are at their minimum possible size, everything is embedded in a ScrollView.

So far, I've been able to use SingleChildScrollView to enable scrolling only if the views overflow the parent widget's height. Independent of that solution, I've been able to use ConstrainedBox along with the Flexible widget to get a view to shrink as necessary to fit in the available space.

return SafeArea(
    child: Column(
         children: [
              const SizedBox(
                  width: 200,
                  height: 400,
                  child: ColoredBox(color: Colors.blue)
              ),
              Flexible(
                child: ConstrainedBox(
                  constraints: const BoxConstraints(
                    minHeight: 50,
                    maxHeight: 800,
                  ),
                  child: Container(color: Colors.red)
                ),
              )
         ],
    )
);

However, I can't get them to work together in a single example. Is this something that's possible with Flutter?