Local-First

One of the hardest problems in local-first software is sync. In previous posts, I talked about OTs/CRDTs, the role of the cloud relative to the client, and general conflict resolution. But I never quite landed on the core theme behind those ideas; sync is the bread and butter of data reconciliation in local-first software. Briefly, a sync engine is responsible for collecting changes, resolving conflicts and propagating these changes across all clients. Considering that every client operates on their own copy of the data, and can go offline for long, unknown stretches of time, a good sync engine must be designed for availability, partition tolerance, and eventual consistency the classic trifecta of trade-offs in distributed systems. ...

The goal of every data reconciliation algorithm is to ensure all replicas converge on the same state of the world, in spite of concurrent changes. The best way to achieve this is through a deterministic merge where the same result can be guaranteed regardless of the order or timing of updates. The backbone of a deterministic merge is one that is associative (grouping doesn’t matter), commutative (order doesn’t matter) and idempotent (reapplying changes yields the same result). ...

When it comes to data reconciliation in the context of building real-time, collaborative user experiences, two major techniques are often discussed: Operational Transform (OT) and Conflict-free Data Replicated Types (CRDT). OT was developed in the late 80s and pioneered the foundational principles of real time collaborative editors. CRDTs emerged much later (around 2006), and were partly motivated by the complexities of implementing OT and the correctness pitfalls it can introduce. If we compare the two methodologies more broadly, OT is a strictly operation-based approach to data reconciliation while CRDTs offer both state-based and operation-based approaches. Examining the two from this lens, it’s clear that OT and CRDT are not simply opposites. More accurately, they are representative examples within a broad spectrum of reconciliation strategies. A more meaningful comparison therefore would be state-based vs operation-based approaches. This better captures the fundamental differences in how changes are propagated and merged across replicas. ...

In my last post, I argued that conflict in local-first is an inherently human problem. Without understanding the intent, and expectations of the end user, reconciliation is not always meaningful, given that correctness doesn’t equal usefulness from a user perspective. Before we dive into the technicalities of how conflicts get resolved, it’s worth exploring why drift happens in the first place and the basics of how it gets resolved. When we work in settings where multiple replicas can be edited independently, conflict is a direct, unavoidable result. Replicas can drift apart for several reasons: network latency, concurrent changes, and offline edits are all a natural cause of divergence in local-first systems. Once the replicas diverge, the need to agree on consistent state arises. The goal here is deterministic reconciliation. In other words, given a set of changes, all replicas should eventually all converge on the same result. ...

Conflict is a recurring topic of discussion in the local-first space. This is unsurprising given that real time collaboration and intermittent offline access inevitably introduces drift between replicas. When multiple users collaborate on a single application or even when a user works on the same application from separate devices, change conflicts occur. This echoes a foundational principle in distributed systems: when networks partition, systems must decide how and whether to be consistent or available. In fact, much of the conversation around conflict resolution in local-first draws directly from distributed systems. Topics such as Conflict-free Replicated Data Type (CRDT) and Operational Transform (OT) all stem from key research in distributed systems. However, unlike in distributed systems where conflict is a matter of direct data reconciliation, conflict in local-first is a result of collaboration and is therefore, arguably, a human problem as much as a technical one. ...

With the rise of mobile in the early aughts, where updates and collaborative edits felt instantaneous (since they wrote directly to local storage and used shared network drives) the traditional client-server model of the web proved clunky. Instead of instantaneous updates that were common to mobile interactions, web updates incurred added latency from the extra round trip it had to make. The responsiveness of native apps set a new precedent for user experience. ...

There’s nothing quite like the clarity of losing your job on a friday that forces you to contend with the bigger question; who and what kind of future do you want to build for? For the last year, I’ve been deep in the local-first ecosystem, learning alongside peers and building the infrastructure to power local-first software. Amidst the hype of AI and agent-driven code, I’ve witnessed a quiet hum of folks demanding to take back control, of their data, of their tools and of the role software has in their everyday lives. Local-first is not just a software methodology, it is a philosophical movement. It’s about building software for people in the context of where they’re at. ...

As the saying goes there is no cloud, just someone else’s computer. Simplistic as it is, the saying captures a universally accepted reality: app data is better off hosted elsewhere living on someone else’s hardware (I’m looking at you GCP and AWS). The issue with this cloud-first sentiment is that it relegates the cloud as a panacea. Sure, giving control to the cloud frees us from the burden of bootstrapping infrastructure and managing complex ops, but it comes with its own set of tradeoffs. Chief among them? The total loss of control over our own data. Not to mention the added reliance on third-party uptime (looking right at you us-east-1). ...

The rise of the local-first movement was no anomaly. For years, browsers were no match for native environments that offered superior performance and offline capabilities. The gap was painfully obvious, native apps could store gigabytes of data and had access to a whole file system while browsers barely scraped by with megabytes. Modern browsers have since caught up over the last few years and the divide between browser and native environments has significantly blurred. There were many developments that gave rise to this, chief among which are: larger browser storage capacity, and new storage APIs. Let’s examine these. ...

In my last post, I mentioned that what makes an app local-first is its ability to keep running even when a connection falters. At first glance, this might look a lot like the offline-first movement that arose alongside the progressive enhancement wave of the mid to late 2010s. But there’s a subtle distinction. Offline-first apps focused primarily on staying functional during network interruptions but the server remained the primary data source. Data in this context is stored locally until a connection is restored, after which the locally stored data is deleted in favor of the remote store. A restored network connection progressively enhanced the experience and syncing only happened when there was new data created during the interim offline period to upload. ...