Data Retention

You can configure a Backup to delete old data periodically. Currently, we support these retention policies:

• Keep (the default): all data is kept forever;
• Delete: the data is deleted after some duration you provide.

Policies are defined on a backup using a set of labels, the main one being io.kannika/data-retention-policy to select the policy. See details for each policy below.

The ‘Keep’ policy

When using this policy, your data won’t be deleted at any point. This is the default policy if none is defined, but if you wish to set it explicitly, then set the io.kannika/data-retention-policy to Keep.

apiVersion: kannika.io/v1alpha
kind: Backup
metadata:
  name: backup-example
  labels:
    io.kannika/data-retention-policy: Keep
spec:
  source: "my-eventhub"
  sink: "my-storage"
  streams:
    - topic: "active-stream"

The Delete policy

The Delete policy will mark Kannika Armory segments as expired after some duration of your choosing. The expiration date of a segment is computed using the latest record timestamp it contains.

Once a segment’s expiration date has passed, it becomes eligible for deletion. See How expired data is removed for when that actually happens, and Changing the retention policy on an existing Backup for adjusting the duration later.

The duration is expected to match this regular expression: ([0-9]+(ns|us|ms|[smhdwy]))+.

apiVersion: kannika.io/v1alpha
kind: Backup
metadata:
  name: backup-example
  labels:
    # Mark segments as expired after 30 days, and remove them.
    io.kannika/data-retention-policy: Delete
    io.kannika/data-retention-policy-delete-after: 30d
spec:
  source: "my-eventhub"
  sink: "my-storage"
  streams:
    - topic: "active-stream"

How expired data is removed

While the Backup is running, a background task (the segment reaper) periodically checks each topic for expired segments and deletes them from the storage. If the Backup job is stopped, expired files are not removed until it runs again, though they will be ignored by an eventual Restore in the meantime.

Deletion is not immediate. After a segment expires, the reaper leaves it untouched for a grace period of one hour so it is never removed while a Restore might still be reading it. It then schedules its next check from the earliest upcoming expiration, checking at most once an hour when nothing is close to expiring and more frequently as an expiration approaches.

When the reaper removes a segment, it logs a line like this at the info level:

{"timestamp":"2026-07-24T09:12:44.512013Z","level":"INFO","message":"📅 Segment has expired","segment_id":"01JZ8F3K2PQ4RN6S7T8V9WXYZ0"}
{"timestamp":"2026-07-24T09:12:44.698215Z","level":"INFO","message":"📅 Segment has expired","segment_id":"01JZ8F5M1H3B4C5D6E7F8G9H0J"}

At the debug level, it also logs when it schedules its next check:

{"timestamp":"2026-07-24T09:12:44.698471Z","level":"DEBUG","message":"Next expired segment check in 3603s"}

See Logging for how to adjust the log level and format.

Changing the retention policy on an existing Backup

You can set or change the retention policy at any time, either from the Console or by editing the labels on the Backup resource directly. This works the same whether the Backup currently uses the Keep policy or an existing Delete policy.

Segments that have not expired yet get their expiration date recalculated against the new duration. Shortening the duration therefore pulls more segments into scope for deletion. The reaper removes them the next time it runs, so the Backup job needs to be running for the expired files to actually be deleted.

Removing a topic’s data in one go

A retention policy is the right tool for deleting old data. If instead you want to remove a topic’s backup entirely as a one-off, delete that topic’s directory directly from the storage. See Deleting a topic’s data for the procedure.

Note

Deleting a topic’s backup data from the API and the Console will be supported in a future release.