upgrade Flink minor version and restore from checkpoint

119 views Asked by At

From official doc, it says Flink support minor version upgrade - restoring a snapshot taken with an older minor version of Flink (1.x → 1.y)..

Q1. Does it means I can upgrade Flink version of my job in following way:

  1. Stop job running with Flink 1.10.
  2. Record latest checkpoint, for example: chk-123.
  3. Upgrade Flink to 1.15(or higher).
  4. Restore the job with chk-123.

Q2. I found there is savepoint compatiabiliy table, but checkpoint is not mentioned. Is checkpoint compatiabiliy table the same as savepoint, or just as described as Flink (1.x → 1.y) (x means any version previous than y)?

2

There are 2 answers

1
Dominik Wosiński On BEST ANSWER

I think it should work the same way for externalized checkpoint as for savepoint in terms of compatibility. The thing is that savepoints are done automatically when You stop the job, so they are a natural pick when updating a version of Flink to minimize the amount of data reprocessed. Is there a reason why You can't use savepoint?

0
Martijn Visser On

This depends on what type of snapshot (canonical savepoint, native savepoint, aligned checkpoint or unaligned checkpoint) you're using and if you are changing your Flink application of course.

You can find the overview for these snapshots and the capabilities and limitations they offer at https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/ops/state/checkpoints_vs_savepoints/#capabilities-and-limitations