Flutter: redraw Custompaint lines after changing scale

Question

I am creating a note-taking app using Flutter and I use a CustomPainter for the drawing part. For better performance, the CustomPaint gets converted into an image after some Lines. Now the problem with images is when zooming in, the lines (which are Images now) are very pixelated. Now my first Idea to solve this was redrawing the lines after zooming.

But how do I redraw them correctly without them still having a bad resolution? Also, is it possible to only redraw the lines that are visible on the screen currently?

everything is wrapped with a layoutbuilder which provides constraints + pixelratio (scale):

    LayoutBuilder(builder: (context, constraints) {
          size = constraints.biggest;
          scale = MediaQuery.of(context).devicePixelRatio;

How I did the zooming:

                  GestureDetector(
                    onScaleStart: (details) {
                      _initialFocalPoint = details.focalPoint;
                      _initialScale = _scale;
                    },
                    onScaleUpdate: (details) {
                      setState(() {
                        _sessionOffset =
                            details.focalPoint - _initialFocalPoint;

                        _scale = _initialScale * details.scale;
                      });
                    },
                    onScaleEnd: (details) {
                      setState(() {
                        _offset += _sessionOffset;
                        _sessionOffset = Offset.zero;
                      });
                    },
                    child: Transform.translate(
                        offset: _offset + _sessionOffset,
                        child: Transform.scale(
                          scale: _scale,
                          child: .... 

my CustomPaint which comes somewhere after the Transform widgets:

                               CustomPaint(
                                  size: size!,
                                  willChange: true,
                                  painter: DrawingViewPainter(
                                      pointsList: drawingNotifier.points,
                                      image: image
                                  ),
                                );

snippet with the custpmpaint to image converter (drawn lines get converted to images to save performance):

  ui.PictureRecorder recorder = ui.PictureRecorder();
  Canvas canvas = ui.Canvas(recorder);
  canvas.scale(scale!);

  DrawingViewPainter(image: image, pointsList: points).paint(canvas, size!);
  ui.Picture picture = recorder.endRecording();
  ui.Image newImage = await picture.toImage(
    (size!.width * scale!).ceil(),
    (size!.height * scale!).ceil(),
  );
......
image = newImage

the Custompaint part with the image drawer:

canvas.drawImageRect(
          image!,
          Offset.zero & Size(image!.width.toDouble(), image!.height.toDouble()),
          Offset.zero & size,
          Paint());
    }

(sorry for the huge amount of code)

If you know of a better way please let me know as well.

Here a video of the Problem