StateVec is a deterministic, verifiable transaction engine for core business state: balances, ledgers, positions, limits, billing, reservations. You define the data model and write the business rules as a deterministic plugin. StateVec gives you atomic in-memory transactions, one Raft-ordered history replicated across nodes, and recovery from that same history.
The idea fits in one sentence: the same commands, applied to the same starting state with the same plugin version, must produce the same results on every replica, during live execution, during replay, and during recovery. Everything else in this article follows from making that hold and checking that it holds.
Why a transaction core
When one business action touches state owned by several services (order → account → limit), the system has to handle distributed transactions, compensation, duplicate requests and partial failure. Coordination ends up on the critical path, and tightly coupled writes are spread across team boundaries.
StateVec takes the opposite approach for the state that really has to be consistent. Keep tightly coupled changes inside one transaction core, and let everything else stay loosely coupled around it. Gateways, microservices, query stores and notification systems remain outside. They consume committed facts, and they influence the core only by submitting new commands.
That boundary is deliberate:
- Consistency covers the business state managed by one StateVec cluster. Payment channels, other databases and other clusters need their own integration protocols.
- All replicas share one write history. Sharding writes across clusters requires a separate design.
- Transaction handlers stay short and deterministic. Long computations, external HTTP calls and analytical queries belong outside the core.
Who owns what
The architecture is easiest to understand as a set of owners, each with exactly one job. No owner borrows another's authority.
| Responsibility | Owner |
|---|---|
| Records, commands, events, business rules | Your business plugin |
| Overlay simulation, results, state installation, execution evidence | Engine |
| Term, vote, membership, replication progress, commit | raft-rs, called only by the Raft owner |
| Admission, durability/apply joins, checkpoint adoption, catch-up | Raft owner (RaftOwner) |
| Choosing and validating the exact local root before service | Startup owner (StartupOwner) |
| Sockets, files, engine work | Bounded physical ports and workers |
Workers and ports do physical work and return a typed result. They never decide retry, currentness or recovery. Metrics and console endpoints read projections and cannot trigger a state transition. This shape keeps every semantic decision in one place, and lets the same production code run under deterministic simulation tests.
Modeling: records, commands, events
Developers describe the business in three kinds of types. Records are the stored state, commands are requested actions, and events describe business changes. Rust macros generate typed accessors, full request validation, key and index encoders, and a schema fingerprint from those definitions.
A few rules keep handlers replayable:
- Structure is validated before business logic. Generated validation checks field lengths, enum values, fixed-width encoding and trailing bytes. Handlers only enforce business rules.
- No hidden inputs. Handlers must not depend on external I/O, unconstrained randomness or the machine's clock. Time enters as an explicit, replicated reference time (
TxAccess::ref_tx_time_ns()). Simulation, verification, apply and replay all see the same value. - Business rejections are typed. Codes
1..=59999belong to your plugin and are bound to its execution revision.60000..=65535are reserved for the framework.
Read the plugin contract for the full developer boundary.
Transactions: ACID over one ordered history
Each command is its own transaction. The engine runs it against a simulation overlay: it reads the immutable canonical base plus the effects of earlier successful commands in the same batch. No full copy of the base is needed. The stacked data view explains how reads see staged changes.
- Atomicity. A command's changes apply completely or not at all. A business rejection has no business delta, but it still consumes its position and its chain step. It neither rolls back earlier commands nor cancels later ones.
- Consistency. Requests are validated against the typed model before admission. Deterministic plugins give every replica the same result, and Verify mode checks this before commit. Declarative business-invariant checks are planned for V1.
- Isolation. Commands execute one at a time in one global order, which gives serializable isolation. A single-threaded engine worker per node applies the order that the Raft log fixes on every replica.
- Durability. A command is committed only after it is replicated and persisted with the required
fsynccalls on a quorum of voters. There is no mode that disables sync and no group commit that defers writes across batches.
Batching is the main throughput lever: one Raft entry carries many commands, so replication and sync costs are shared. A batch is not one big transaction. Grouping the same commands in batches of 16 or 1,024 yields identical per-command outcomes. Results become visible together, and the applied frontier never advances part-way through a group.
A panic or an access or host failure during preparation is not a business rejection. It abandons the overlay. On a committed entry, such a failure stops the node rather than being skipped or turned into a client success.
Verify mode and Performance mode
Raft decides which commands commit and in what order. The execution mode decides when business logic runs relative to commit, and whether followers check it.
Verify mode is the default. The leader simulates the batch from a known base and computes deltas, events, result digests and chain hashes as declarations. Business state is not touched. Voting followers re-execute the same commands from the same base, compare against the declarations, and complete the required log writes and syncs before acknowledging. After commit, each node applies locally. A node can reuse its retained preparation only when the base, inputs and declarations match exactly. Otherwise it executes and checks again.
Performance mode is an explicit alternative. Commands pass static preflight, replicate with the same Raft durability, and execute in order at committed apply. They still produce real results and a real hash chain. Performance mode omits only the pre-commit comparison.
| Verify mode | Performance mode | |
|---|---|---|
| Business execution before commit | Leader simulation + follower verification | None (static preflight only) |
| Raft durability and quorum | Required | Required |
| Recovery | Checkpoint + committed suffix | Checkpoint + committed suffix |
| Detects replica divergence | Before commit | Via chain evidence after apply |
| Fallback between modes | Never automatic | Never automatic |
A mismatch in Verify mode is a fault, not permission to downgrade. One configured exception exists: a learner that the initial configuration permanently bars from promotion may acknowledge durable replication before executing. Its ACK never counts toward the quorum, and it still validates every committed command. A learner that may later be promoted verifies like a voter.
Commit and apply are distinct. Commit means a quorum durably stored the command at a fixed log position. Apply means a node executed it and installed the business state. Clients receive final results only after apply.
Verify mode catches divergence between replicas. It cannot catch a business-logic defect that every replica shares, and it does not defend against malicious replicas. A single-voter cluster has no independent verification.
Reads, views, and audit
The core makes decisions. Surrounding systems build views.
- Inside execution, records support direct access, unique-key lookup and deterministic ordered indexes. Reads see staged changes from earlier commands in the batch.
- Outside the core, read-side projectors turn committed transactions into query views, notifications and real-time decision inputs. The projector plugin framework is implemented. Its integration with V1 non-voting learners is pending.
- Views can lag and never authorize or overwrite core state. A projection that wants to change something submits a new command, and the core validates it. Rules that gate admission must run in the core.
- A recovery checkpoint is not an audit archive. Retaining or exporting command history for audit is a separate operating policy.
Clients and responses
Each ingress client keeps one outstanding batch and retries it with the same identity, trying each configured node in turn. While commit status is unknown, it resends. Once commit is confirmed, it only queries the result. A disconnect does not cancel server-side work.
Live response bodies and events are not durable. A restart or snapshot recovery can lose a cached result even when execution can still be confirmed. Applications that must survive their own crash should persist, or be able to reconstruct exactly, each request's original bytes, identity and grouping.
Where it fits
StateVec suits workloads with well-defined rules, bounded core state, and short, deterministic, replayable operations:
- Financial core. Spot and margin trading, order checks, positions and funds changing in one atomic transaction.
- Accounting. Ledgers, balances and settlement state, with explicit rejection rules and no one-sided entries.
- Billing and limits. Usage, pricing, available limits and credit deductions. A retried request with the same identity is not charged twice.
- Reservations. Ticketing and inventory, with reserve, confirm and release as explicit commands, and expiry driven by replicated time.
It is not a general-purpose database. Reporting and ad-hoc queries belong in read models. External delivery belongs in queues. Writes need a quorum, so a network partition can make the core unavailable for writes rather than inconsistent.
Continue with Raft history and recovery or the plugin contract.