I'm evaluating SageMaker feature store but am finding that custom backfilling doesn't appear to be straight forward. I need to be able to backfill new features that are added to feature groups. If I add a new feature to a feature group I will have an incomplete table for that feature.
| record id | event time | old feature | new feature | write time | is deleted |
|---|---|---|---|---|---|
| 0 | timestamp | data | nan | timestamp | false |
| 1 | timestamp | data | nan | timestamp | false |
| 2 | timestamp | data | data | timestamp | false |
My first though was to delete the incomplete records and then insert the new backfilled record. But if I delete the record id 0 or 1 from the offline store it will insert a gravestone record to show delete time rather than hard delete the original row.
| record id | event time | old feature | new feature | write time | is deleted |
|---|---|---|---|---|---|
| 0 | timestamp | data | nan | timestamp | false |
| 1 | timestamp | data | nan | timestamp | false |
| 2 | timestamp | data | data | timestamp | false |
| 0 | timestamp | timestamp | true | ||
| 1 | timestamp | timestamp | true |
How can I backfill individual features?