Crate layout

The workspace, what depends on what, and where the language-neutral half ends.

The crates

crate	role
kladde-store	the storage layer: the file, pages, the address table, the journal, the flush, and consolidation, behind the Backend traits. Type-agnostic; knows nothing about serialization.
kladde-persist	the serialization layer: Persistable, Guard, Location, the scalar guards, and the schema binding.
kladde-schema	type descriptors, their canonical encoding, and fingerprints. Depends on nothing but kladde-varint.
kladde-varint	LEB128 varints.
kladde-types	the built-in containers: vector, hash map, string, blob. The default collection, not a layer — it uses only the public API any library could.
kladde-derive	the #[derive(Persistable)] macro. Reached through kladde’s derive feature; applications do not depend on it directly.
kladde	the application-facing entry point: creating and opening files, the root value, transactions and batches. Re-exports the derive macro and everything a Persistable impl names, so an application needs this crate and nothing below it.

The dependency direction is strictly downward, and kladde-store deliberately depends on nothing but kladde-varint — it is usable on its own as a persistent store of byte allocations, independently of anything above it.

graph TD
  APP["<strong>application code</strong>"]
  K["<strong>kladde</strong><br/>Kladde, Transaction, Batch"]
  T["<strong>kladde-types</strong><br/>containers"]
  D["<strong>kladde-derive</strong><br/>derive macro"]
  P["<strong>kladde-persist</strong><br/>Persistable, Guard, Location"]
  S["<strong>kladde-schema</strong><br/>descriptors, encoding, fingerprints"]
  V["<strong>kladde-varint</strong><br/>LEB128"]
  H["<strong>kladde-store</strong><br/>Store, Pointer, journal, pages,<br/>Backend / ReadBackend / WriteBackend"]
  APP --> K
  APP -. optional but common .-> T
  K --> H
  K --> P
  K -. "derive" feature<br/>(on by default) .-> D
  T --> P
  D --> P
  P --> H
  P --> S
  S --> V
  H --> V

The parts

kladde-store — the storage layer

Owns the file. It implements everything the implementation notes describe, and adds only what Rust needs: the Backend traits the layer above calls, the Store that implements them, and a Storage trait with two implementations, a real file and a byte vector in memory. See The store for the Rust-specific half.

kladde-persist — the serialization layer

Where types meet bytes. Persistable, its Guard counterpart, Location, and the guards of the scalar types. Knows about the store’s traits; knows nothing about any particular container or derived type.

kladde-schema — type descriptors

The Rust implementation of the language-independent schema model. It depends on neither the store nor Persistable, deliberately, because it must be portable and testable on its own; the binding between a Rust type and its descriptor lives one layer up, in kladde-persist. See Schema binding.

kladde-types and kladde-derive — the type vocabulary

The built-in containers, and the macro that turns user types into backed ones. Both build on kladde-persist, and neither depends on the other.

The macro is re-exported from kladde, not from kladde-types, so that an application can use the containers without the macro machinery and vice versa — and because generated code is rooted at ::kladde, which is where the items it names are re-exported from.

kladde — the entry point

Kladde<T> pairs a root value with a Store, and is what application code holds. It writes the root’s descriptor table and fingerprint into the file at creation, checks the fingerprint at open, and hands out guards, transactions, and batches.

The seam

For anyone porting kladde, one boundary matters more than the rest.

Below kladde-persist, the design is language-neutral. The address table, the journal, the flush, the consolidation policy, and the schema model all translate more or less directly; they are documented in Specification and Implementation rather than here.

At and above kladde-persist, the design is Rust-specific and should be redone.

• Persistable/Guard exists because Rust cannot intercept a field assignment.
• The &self write / &mut self read split exists because Rust’s borrow checker can then enforce “no reading stale data mid-write” for free.
• INLINE_SIZE as an associated constant exists because Rust can compute field offsets at compile time. A language without compile-time constant folding will compute offsets some other way — probably once per type at startup, which is fine, but it changes the shape of the code.

A Python port should keep the storage layer almost verbatim and throw away the guards entirely. A C++ one might use proxy objects with operator=; a Java one might use bytecode enhancement, which is what db4o did.

Cross-cutting concerns

Crash consistency is a property of the journal and the flush, but every container must uphold the ordering discipline for it to hold, and no type system enforces it.

Freeing is a type-driven hook that spans the persistence layer and the store; see Freeing.

Schema evolution touches the load path, the mutation path, and the flush, because a value read at a foreign layout must not then be mutated at native offsets; see the specification. Until it exists, a file whose root fingerprint differs from the application’s is refused at open.