Samza provides build-in support for persistent state store, backed by RocksDB on the disk and changelog in Kafka. There are many possible usage scenarios that users want to have various different features to be enabled. This guide attempts to give a general suggestion to various configuration of RocksDB state store in different application scenarios.
Supported Persistent KeyValue Store Use Cases
| changelog | TTL | Host-affinity | Description |
|---|---|---|---|
| No | No | No | Not recoverable local state stores, will lose all data when container restarted |
| Yes | No | No | Recoverable local state stores, keeps old data, re-bootstrap when container restarts |
| No | Yes | No | Not recoverable local state stores, expiring old data, will lose all data when container restarted |
| Yes | Yes | No | Recoverable local state stores, expiring old data, re-bootstrap when container restarts (see note for TTL) |
| No | No | Yes | (UNSUPPORTED) Not recoverable local state stores, re-use local state with best-effort when host-affinity succeeds, otherwise losing all data |
| Yes | No | Yes | Recoverable local state stores, keeps old data, quick-recovery when host-affinity succeeds |
| No | Yes | Yes | (UNSUPPORTED) Not recoverable local state stores, expiring old data, re-use local state with best-effort when host-affinity succeeds, otherwise losing all data |
| Yes | Yes | Yes | Recoverable local state stores, expiring old data, quick-recovery when host-affinity succeeds (see note for TTL) |
...