Zeebe is a horizontally scalable, language agnostic, highly available, process orchestration engine that can execute a BPMN 2.0 workflow.
Key aspects of Scalability
- No central database
- Event sources (append only logs). Logs are written to disks
- Log replication (Atomix) and Compaction (delete event after all exporters have processed an event).
- RocksDb for snapshots / Projections.
- Support for many exporters like Elasticsearch, Kafka and Hazelcast.
- One thread per log means no locking contention.
- Peer to peer clusters.
- Requires 2 nodes initially (main and contact point), then GOSSIP (that’s how EPIDEMICS spread).
Replication using RAFT consensus algorithm with ATOMIX as the implementation. http://thesecretlivesofdata.com/raft/
Leaders can only write, followers are for replicas.
Zeebe is a CP system in the CAP theorem.
To achieve consistency, QUORUM must be obtained.
[ quorum ≥(replication group size / 2) + 1 ]
QUOROM avoids split-brain phenomena.
- Partitions for throughout and multi-threading:
- Each partition is a separate physical append only log.
- Each partition has its own leader that gets write access.
- All workflow events of a single flow go to the same partition for single write access
- Use of gRPC for remote communication.
|||
