Skip to content

Concepts

Question this section answers: Which piece do I touch for storage, buffering, or offsets?

object-log is three cooperating surfaces:

┌─────────────┐     ┌────────────┐     ┌─────────────┐
│  BlobStore  │◄────│ LogEngine  │────►│  Sequencer  │
│  (bytes)    │     │ (buffer)   │     │  (offsets)  │
└─────────────┘     └────────────┘     └─────────────┘

What stays outside

ConcernOwner
Kafka wire / record batch codecConsumer protocol crate (e.g. heimq)
Producer id / epoch / sequenceSequencer::Meta in the consumer
Epoch fencing / EOSSequencer implementor
Hot-tier local fsyncConsumer buffer in front of object-log
Retention policyConsumer; object-log supplies truncate_before mechanism