Scaling CALayer without altering sublayer transform

Question

I have a CALayer called photoLayer which has a sublayer called imageLayer whose contents is a CGImageRef. The user can select a crop rect in the image & also zoom with fingers using Pan/Zoom gestures (this selection is part of another UI using UIScrollView). To display the image in imageLayer with chosen crop rect and zoom value, I apply a scale transform to imageLayer to chosen zoom value. I also adjust imageLayer's position property to match the top left corner in the image crop rectangle selected using pan/zoom. This way I am able to display imageLayer with cropped/zoomed image correctly.

But now I also need to scale up photoLayer (i.e. parent layer of imageLayer) around the centre and when I apply the transform.scale animation to photoLayer, the imageLayer does not scale along the centre. I was expecting imageLayer to be unaffected by super layer's scale animation but that's not the case. What am I doing wrong ? Here is how I scale photoLayer :

            CABasicAnimation *scale = [CABasicAnimation animationWithKeyPath:@"transform.scale"];
            [scale setFromValue:[NSNumber numberWithFloat:0.0f]];
            [scale setToValue:[NSNumber numberWithFloat:1.0f]];
            scale.beginTime = startTime;
            scale.duration = 1.f;
            [scale setRemovedOnCompletion:NO];
            [scale setFillMode:kCAFillModeForwards];

            [photoLayer addAnimation:scale forKey:@"scaleUp"];

I really need imageLayer to follow photoLayer's scale animation around centre.