Partial sync

This guide covers subset mode, where a client holds only what its live queries need. You will learn:

• The measured trade between the two sync modes
• How a live query becomes a subset, and who decides
• Why a row leaving a view is a forget and never a delete
• How subsets are shared across tabs and released
• When not to use this

Status: Built. The API below exists and is under test. The measurements below are the prototype's, and have not been retaken against this engine.

Overview

In partition mode a client replicates its whole partition, which is what makes offline reach unconditional: every row you are allowed to see is readable and writable with the network gone.

That stops working at some size. Not sharply — gradually, which is worse, because it looks fine in every test small enough to run quickly.

subset mode syncs only what open queries ask for. It is three orders of magnitude cheaper on every axis except the one this library exists for, and the guide is mostly about that exception.

The measurement

Taken on the prototype against an issue tracker of 1,220,358 rows, one project of 427,728 rows, using the real engine rather than a model.

Read the last two rows before the others. A partition client has the whole project to work against with no network. A subset client has what it happened to load — two hundred issues, exactly one of which has its comments.

An application choosing subset is choosing to be an online application that survives a disconnection, rather than a local-first one. That is a legitimate choice and it is not the same product.

Choosing a mode

export const replica = createReplica({
  schema,
  server: '/sync',
  store: stores.opfs(),
  mode: 'subset',            // default: 'partition'
})

Declared at construction and never inferred. A client that pulled its partition on startup and narrowed afterwards has already paid the cost it was avoiding, and would look correct in every small test — so the mode is a property of the replica, not a description of what happens to be open.

The mode is part of the fingerprint. Changing it is a different store.

How a query becomes a subset

You do not open subsets. TanStack DB pushes each live query's where, orderBy and limit down through onLoadSubset, and @locel/tanstack turns what it can into a subset:

const { data } = useLiveQuery((q) =>
  q.from({ issue: issuesCollection })
    .where(({ issue }) => eq(issue.projectId, 'roci'))
    .where(({ issue }) => eq(issue.status, 'open'))
    .orderBy(({ issue }) => issue.modifiedAt, 'desc')
    .limit(50)
)

That becomes one subset: two equality filters, an ordering, a cap of 50. The server sends those rows and keeps them live.

The division of labour is what keeps this small. TanStack DB decides which subsets should exist — it parses the queries, deduplicates overlapping ones and tears them down when nothing is watching. locel decides what is in them and keeps them current. Neither half reimplements the other.

Pushdown is deliberately partial. eq, gt, gte, lt, lte and and are extracted; everything else stays in TanStack DB's own predicate, which runs over the rows locel delivers. So a filter locel does not understand costs a wider subset — more rows synced than strictly needed — and never a wrong result. Guessing at an operator is the failure that would be wrong, so anything unrecognised is dropped rather than approximated.

Under or, neither branch bounds the result, so nothing is extracted at all.

Subsets need indexes

Every subset is answered by an indexed query on the server, on every commit that touches the collection. Declare the indexes in the schema:

indexes: [['projectId', 'status'], ['projectId', 'modifiedAt']],

A subset with no index that can serve it is refused at open with E_MISSING_SUBSET_INDEX. A partial-sync deployment that silently full-scans on every write is worse than one that refuses to start, because the first symptom arrives in production under load.

The awkward part of that is discovering it late: you write a live query, and the refusal arrives the first time a component mounts. So in development the adapter prints the declaration you are missing rather than only the error —

[locel] issues: no index serves { projectId, status } ordered by modifiedAt.
        Add to the collection:  indexes: [['projectId', 'status', 'modifiedAt']]

— which is the same check, moved to where you can act on it. There is deliberately no way to declare subsets ahead of time to get this at boot: the whole reason subsets stay small is that the queries decide what exists, and a second place to declare them would be a second thing to keep in step.

Eviction is not deletion

Close an issue and it leaves "open issues" while remaining a perfectly live row.

A client that recorded that as a delete would write a tombstone into its own durable store and hide a live row after every reload. So the change stream carries evictedFrom — which carries no data — and the replica records a forget: the row leaves the view, and any pending write against it stays in the outbox.

You will not usually observe this. It matters when you are reading the local log in a bug report, and it matters because it is the reason subset mode does not slowly corrupt a store over a long session.

Sharing and release

Subsets are owned by the broker, so two tabs running the same query share one subset and one server-side registration. The cross-tab case is not a special path — a tab does not open storage or a connection of its own, so there is nothing to reconcile.

Releasing is deferred:

createReplica({ /* ... */, mode: 'subset', subsetTtl: 30_000 })

A released subset stays live for subsetTtl — still subscribed, still current — so returning to a view is instant. When the timer expires it is closed for real, and the rows only it was holding are dropped, reference-counted against every subset still open.

Without the TTL, browsing four views took a client from 230 rows to 430 with nothing to stop it. With it, 230 to 230. The cost lands on the other side of the ledger: offline reach fell from 400 readable issues to 200, because a released view is now genuinely released.

Membership is evaluated client-side with the same function the server runs. Two implementations of "does this row belong" would disagree eventually, and the disagreement would be a row stranded in a view or missing from one.

Windows are re-queried, not maintained

orderBy with limit — "the fifty most recently modified" — is the one subset shape whose membership changes without the row changing. Somebody else's write can displace a row nobody touched.

Maintaining that incrementally means knowing which row outside the window takes the place of a displaced one, and that is a row the server is not tracking. Getting it wrong loses rows silently and permanently.

So a window is re-read and diffed against what the client was last told: one indexed, capped query per affected commit. The cost is bounded and measurable rather than hidden. This is the part a full incremental-view-maintenance engine does better, and saying so is cheaper than pretending otherwise.

What the server holds

Per-subset membership is tracked server-side, per connected client.

The first implementation evicted any changed row that failed a filter, whether or not the client held it — which sends a frame to every client for every write anywhere in the partition, destroying the bandwidth win entirely. Tracking the key set per open subset is what makes the numbers in the table real.

It costs memory bounded by what clients hold: connected clients × open subsets × rows in view. A partition deployment holds none of that. Budget for it before choosing subset mode at scale.

When not to use this

• The product promises offline work. Not "keeps rendering" — actual work, on data the user did not think to open first. Use partition.
• Your partition is already small. Under roughly ten thousand rows, partition mode holds everything for less than the machinery of subsets costs.
• You cannot index the filters. Every subset needs one.
• Writes are frequent and scattered across a large partition. Every commit costs a membership evaluation per open subset.

The honest summary: subset mode buys a dataset a replica cannot hold at all, and pays for it with the guarantee that made you choose a local-first library. Choose it when the dataset genuinely does not fit — not to make startup faster.

Where to go next

• Collections — declaring the indexes subsets need
• The replica — who owns subsets, and reference counting
• Partitions — the boundary subsets narrow inside