How to expose mutable list as a readonly property in a MonoBehaviour

Question

Let's say I have something like this

public class BodyPart : MonoBehaviour {
    private BodyPart parent;
    private List<BodyPart> children;
}

and I want to be able to maintain both links when adding/removing children and changing parents. Ideally I would just create properties, and ensure parent gets updated when I add children and vice versa. But exposing list as a property (even a readonly property) - allows me to do

bodyPart.Children.Add(new BodyPart());

which won't call the setter, so parent field won't be updated properly.

I've seen people exposing List<...> as a property of type ReadOnlyCollection<...>, but I'm not sure if unity component system will work with such a property, and maybe there is some better way? Also - maybe IReadOnlyCollection is better?

I know I could do a hierarchy of GameObjects instead, and use the builtin parent-child relationship of unity, but I don't want my BodyParts to be full-blown GameObjects with transforms and everything, it's just a model for RPG attributes and inventory system. Also the example is simplified and ultimately I want to have more logic and several kinds of relationships between Components encoded in that model.