Quickstart for Node

Python · Node · Go · Rust · Swift · Dart · JVM · .NET · C++ · Objective-C

This program shows two devices synchronizing through one remote. It resolves a conflict so both devices finish with the same row. See Concepts for the database and synchronization terms.

Term	Meaning in this program
Local store	The database files on one device.
Remote	The shared copy of a database that devices synchronize with.
Revision	One published database state that does not change.
Head	The remote’s newest published revision.
Push	Send local writes to the remote.
Pull	Receive writes from the remote.
Pending	Local writes that the device has not confirmed as published.
Replay	Run local statements again on the remote revision.
Quarantine	Preserve a transaction that cannot replay safely, for the application to handle.

The two databases represent a laptop and a phone. Both change the same row before receiving the other’s change. The remote uses a temporary directory that the program removes when it exits.

1. Complete the installation.
2. Save this program as quickstart.js.
3. Run node quickstart.js.

The program prints two lines:

Output	Meaning
Use query for this statement.	The program passes a read statement to execute on purpose. The call fails with read_in_execute, and the program prints the error’s action.
quickstart: ok	The program completed all checks, including checking that both devices contain the same row.

Without an accepted license key, a trial notice on standard error is expected. See license notices for the text. If the program fails or omits these lines, check the installation and any changes to the program.

// Two devices, one remote, one conflict, and its resolution.

const assert = require("node:assert/strict");
const { EventEmitter, on, once } = require("node:events");
const fs = require("node:fs");
const os = require("node:os");
const path = require("node:path");
const potluckdb = require("@potluckdb/potluckdb");

// One "pushed" is one automatic cycle's work, not proof that everything written
// so far is published; a device is finished when it has nothing pending. `on`
// buffers from the moment it is called, so a cycle that completes between a
// status read and the next wait is not missed.
async function waitUntilPublished(device, events) {
  const pushes = on(events, "pushed");
  try {
    while ((await device.status()).pendingWrites > 0n) await pushes.next();
  } finally {
    await pushes.return();
  }
}

async function main() {
  const scratch = fs.mkdtempSync(path.join(os.tmpdir(), "potluckdb-"));
  const remote = potluckdb.Remote.fromDirectory(path.join(scratch, "remote")); // a bucket replaces this in production; see the installation page

  // The laptop synchronizes on its own: automatic is the default policy.
  // onSync delivers every outcome to the program; this program emits each as an event
  // named by its kind and registers a waiter before the write it waits for.
  const laptopEvents = new EventEmitter();
  const laptop = await potluckdb.open(path.join(scratch, "laptop"), remote, new potluckdb.OpenOptions({
    onSync: (outcome) => laptopEvents.emit(outcome.kind, outcome),
  }));
  laptopEvents.on("failed", (outcome) => { throw outcome.error; });

  await laptop.transaction(async (setup) => { // one transaction: both statements travel in one push
    await setup.execute("CREATE TABLE note(id INTEGER PRIMARY KEY, body TEXT NOT NULL)");
    await setup.execute("INSERT INTO note(body) VALUES(?)", ["buy milk"]);
  });
  await waitUntilPublished(laptop, laptopEvents);

  // The phone synchronizes only when instructed: a manual policy, and sync()
  // calls that return the outcome as a value. It starts from the revision the
  // laptop published.
  const phone = await potluckdb.open(path.join(scratch, "phone"), remote, new potluckdb.OpenOptions({
    syncPolicy: potluckdb.SyncPolicy.manual(),
  }));
  const { columns, rows } = await phone.query("SELECT id, body FROM note ORDER BY id");
  assert.deepEqual(columns, ["id", "body"]);
  assert.deepEqual(rows, [{ id: 1, body: "buy milk" }]);

  // Both devices modify the same row before either has seen the other's write.
  await laptop.execute("UPDATE note SET body = ? WHERE id = ?", ["buy oat milk", 1]);
  await phone.execute("UPDATE note SET body = ? WHERE id = ?", ["buy milk and eggs", 1]);
  await waitUntilPublished(laptop, laptopEvents);

  // The phone's push finds a remote head it has not seen:
  // a conflict, returned rather than thrown.
  assert.equal((await phone.sync()).kind, "conflict");
  const resolution = await phone.resolveConflict(); // adopt the laptop's revision, replay the phone's statement on it
  assert.deepEqual(resolution.quarantined, []);
  const pulled = once(laptopEvents, "pulled");
  assert.equal((await phone.sync()).kind, "pushed");

  // The laptop pulls the resolved revision on its own. Both devices hold the same row.
  await pulled;
  for (const db of [laptop, phone]) {
    const { rows } = await db.query("SELECT body FROM note WHERE id = ?", [1]);
    assert.equal(rows[0].body, "buy milk and eggs");
  }

  // query reads, execute writes. The wrong call is refused before execution.
  await assert.rejects(laptop.execute("SELECT count(*) FROM note"), (error) => {
    assert.equal(error.code, "read_in_execute");
    console.log(error.action); // Use query for this statement.
    return true;
  });

  await laptop.close();
  await phone.close();
  await remote.close();
  fs.rmSync(scratch, { recursive: true, force: true });
  console.log("quickstart: ok");
}

main().catch((error) => { console.error(error); process.exit(1); });

What the program demonstrates

The remote uses a directory. No server runs. Remote.fromDirectory uses a directory that stands in for a bucket in a storage service. The installation page shows the constructor for a storage service. These constructors support S3, Google Cloud Storage, and Azure Blob Storage. The rest of the program stays the same.

The laptop synchronizes automatically. Automatic synchronization is the default policy, which decides when to synchronize. The laptop never calls sync(). Its onSync hook, a function that receives synchronization results, reports every attempt. The hook receives an outcome with a kind field. A failed outcome includes the error.

The hook runs on the event loop. You can call the database there. This program converts outcomes into events so it can register a waiter before writing.

Result	Meaning
up_to_date	The attempt found no pending writes and no remote revision to pull.
pushed	The attempt published its pending writes as a remote revision.
pulled	The attempt adopted a remote revision. It can also report replayed and quarantined transactions.
conflict	The device’s push met a remote revision that the device had not seen.
failed	An automatic attempt failed and carries the error.

An automatic result covers one attempt. pushed confirms the writes that the attempt published. Writes made after the attempt began can still be pending. If an earlier attempt failed, an explicit sync() first publishes that attempt’s writes and leaves later writes pending. A program that must know its own writes are published reads status().pendingWrites. waitUntilPublished does that here before the second device opens.

Rows include their column names. columns lists the names. A row is an object with a property for each column name. Integers arrive as numbers. Use readBigInts: true when opening the database to receive bigints.

The phone receives a conflict result. sync() returns conflict when the remote has an unseen revision. This result does not reject the Promise. resolveConflict() adopts the laptop’s revision and replays the phone’s statement. The phone’s text then appears on both devices. The resolution lists any transactions that cannot replay safely as quarantined.

Quarantined transactions

Inspect the quarantined list after every conflict resolution. Quarantined transactions contain local work that the remote does not contain.

query reads and execute writes. Each accepts exactly one statement of its own kind. Each refuses the other kind before execution, with an error whose action names the required call.

These calls promise different things. query returns rows and uses the connection’s maximumRows to bound the result. A different SQL engine can answer a read. execute changes data that a revision publishes and other devices receive.

This execution-intent policy separates reads from writes. SQLite preparation can have effects, including under EXPLAIN.

Neither call accepts SQL transaction control, which starts, commits, or rolls back a transaction.

Use transaction() for transaction control, as the setup shows.

Reference

Open and construct and Database describe every call. Options and policies describes the automatic policy and its parameters. The examples provide longer programs that the test suite runs.