Concepts
potluckdb gives each device a SQLite database. Devices synchronize through an object-storage bucket. This page introduces the model in the order a program meets it. The reference and features pages use these same terms.
One database per device
Section titled “One database per device”The application opens a SQLite database at a path and uses SQLite’s SQL and transactions. SQLite runs unmodified over potluckdb’s virtual file system, with 64 KiB pages.
The local store uses one append-only container per database, on disk or in the browser’s origin private file system. Every commit is durable. One opener owns the container at a time. Each committed write creates an immutable local record containing a portable transaction identity and the pages it changed. Writes never wait for the network.
A bucket, not a server
Section titled “A bucket, not a server”The remote is the shared database copy that devices synchronize with. It holds immutable objects and two small cells, head and floor. The head names the current revision. The floor names the oldest retained revision. Both change only through compare-and-swap, a replacement conditional on their current value.
A directory on disk, an S3-compatible bucket, Google Cloud Storage or Azure Blob Storage can provide the remote. No potluckdb process runs except on the devices.
The objects are pieces, chunks, manifests and revisions:
- A piece is one page, named by its plaintext’s SHA-256 digest.
- A chunk lists a group of piece digests.
- A manifest names the chunks of one database state.
- A revision names its manifest and preceding revision.
Once written, objects never change. The retention policy decides how long they remain.
Publishing
Section titled “Publishing”A push sends local writes to the remote. It seals the local records into one revision. Each changed page becomes a piece. The runtime compresses it, encrypts it when an encryption key is set, and writes it under its digest.
A piece already present under its digest is not replaced. Retrying a push can still make storage requests and transfer bytes.
The runtime installs the revision by replacing head, conditional on it still naming the revision the device last saw. A different head produces a conflict, returned as a value. Bounded uploads per publication keep a fleet’s request rate predictable.
Reading
Section titled “Reading”A database with a remote reads pages on demand. When SQLite requests a page already in the local store, the local store serves it. Otherwise, the runtime fetches the piece by digest, authenticates it and keeps it.
A large database is therefore usable as soon as it opens. The prefetch policy controls how much the runtime fetches ahead of a statement and in the background. Reads wait for the network only when they need a page the device has not fetched.
Synchronization
Section titled “Synchronization”A writable database with a remote synchronizes automatically using the sync policy timers. A manual policy gives scheduling to the program.
One synchronization can exchange data in either direction. If the device has local writes to publish, sync() publishes them. Otherwise, it checks the remote head and pulls a newer revision if one exists. Its outcome is a value: up to date, pushed, pulled or conflict. The automatic driver reports every outcome, including failure, through on_sync.
A pull applies the remote revision’s pages locally without disturbing unpublished local records.
Conflicts
Section titled “Conflicts”A conflict means the device’s push met a revision it had not seen. resolve_conflict downloads and authenticates that revision, then adopts it as the new base. It recognizes already published local transactions by their identities and replays pending statements that are safe to replay.
Statements that cannot replay safely are quarantined. The result names them for the program to handle. Conflict resolution returns a value rather than raising a conflict exception. Before production describes replay limitations that applications must handle.
History and retention
Section titled “History and retention”Each revision is immutable and addressable while retained. open_revision pins one retained revision as a read-only snapshot. It uses the same reads on demand.
Retention keeps the newest revisions. Collection advances floor and removes objects that no retained revision references. A revision behind the floor cannot be opened and reports revision_retired. A database whose base revision is behind the floor also reports revision_retired when a statement needs a piece that the device has not fetched. This continues until the database synchronizes.
Encryption
Section titled “Encryption”When the application sets an encryption key, the runtime encrypts every piece with AES-256-GCM before writing it. A nonce derived from the plaintext digest keeps identical pages identical in the bucket. Encryption therefore preserves deduplication. The bucket holds ciphertext and a key identity.
Each piece’s name is its unencrypted page’s digest. Anyone who can list the bucket learns which pages are equal and can confirm a guess at a whole page. Before production explains this exposure.
admin.reencrypt publishes a retained revision into a new remote under a new encryption key. The advanced page describes the operation.
Analytics
Section titled “Analytics”An analytical read uses DuckDB over a read-only snapshot of the database on the device. Native targets load the engine at run time. On WebAssembly, the engine is linked into the module. The analytics engine page names the library each platform loads.
The same runtime everywhere
Section titled “The same runtime everywhere”One runtime, written in Rust, implements this behavior. Each SDK exposes the same operations. The compatibility page shows where each runs.
| Environment | Runtime access |
|---|---|
| Python, Node and Rust | Generated binding of the Runtime Interface |
| Other native languages | Native Application ABI |
| Browser | Runtime compiled to WebAssembly in a worker |