Simplify readme
This commit is contained in:
parent
de63f2160f
commit
3894e7e401
1 changed files with 37 additions and 56 deletions
93
README.md
93
README.md
|
|
@ -1,8 +1,8 @@
|
|||
# `reconcile-text`: conflict-free 3-way text merging
|
||||
|
||||
Merge a common parent and two edited versions into one text, without conflict markers. `reconcile-text` combines changes from both sides and updates any cursor positions you pass in. It is available as a library for Rust, JavaScript/TypeScript, and Python.
|
||||
`reconcile-text` merges two edited versions of a document using their common parent, without leaving conflict markers. It combines changes from both versions and adjusts any cursor positions you pass in. You can use it from Rust, JavaScript/TypeScript, or Python.
|
||||
|
||||
It is built for syncing text files when all you have are snapshots: the document before editing and the two versions it became. The merge needs no saved operation history or persistent document metadata beyond that parent text.
|
||||
It's designed for syncing text files when all you have are snapshots: the original document and two edited copies. You don't need to record editing operations or keep extra document metadata between merges.
|
||||
|
||||
## Try it
|
||||
|
||||
|
|
@ -10,15 +10,9 @@ It is built for syncing text files when all you have are snapshots: the document
|
|||
|
||||
For the motivation and algorithm behind the library, read [Conflict-Free Three-Way Text Merging][article].
|
||||
|
||||
### Install it in your project
|
||||
|
||||
- `cargo add reconcile-text` ([reconcile-text on crates.io][crates])
|
||||
- `npm install reconcile-text` ([reconcile-text on NPM][npm])
|
||||
- `uv add reconcile-text` or `pip install reconcile-text` ([reconcile-text on PyPI][pypi])
|
||||
|
||||
## Quick start
|
||||
|
||||
Each example merges two changes to the same greeting: one side adds a word, while the other changes the greeting itself.
|
||||
In each example, one edit adds "beautiful" and the other changes "Hello" to "Hi". The merge keeps both changes.
|
||||
|
||||
### Rust
|
||||
|
||||
|
|
@ -28,22 +22,18 @@ Install from [crates.io][crates]:
|
|||
cargo add reconcile-text
|
||||
```
|
||||
|
||||
|
||||
```rust
|
||||
use reconcile_text::{reconcile, BuiltinTokenizer};
|
||||
|
||||
// Start with the original text
|
||||
let parent = "Hello world";
|
||||
// Two users edit simultaneously
|
||||
let left = "Hello beautiful world"; // Added "beautiful"
|
||||
let right = "Hi world"; // Changed "Hello" to "Hi"
|
||||
|
||||
// Reconcile combines both changes
|
||||
let result = reconcile(parent, &left.into(), &right.into(), &*BuiltinTokenizer::Word);
|
||||
assert_eq!(result.apply().text(), "Hi beautiful world");
|
||||
```
|
||||
|
||||
See the [merge-file example][rust-merge-file] for a file-merging CLI, or the [Rust API documentation][rust-docs].
|
||||
The [merge-file example][rust-merge-file] shows how to merge files from the command line. See the [Rust API documentation][rust-docs] for the full API.
|
||||
|
||||
### JavaScript/TypeScript
|
||||
|
||||
|
|
@ -53,14 +43,10 @@ Install from [npm][npm]:
|
|||
npm install reconcile-text
|
||||
```
|
||||
|
||||
Then use it in your application:
|
||||
|
||||
```javascript
|
||||
import { reconcile } from 'reconcile-text';
|
||||
|
||||
// Start with the original text
|
||||
const parent = 'Hello world';
|
||||
// Two users edit simultaneously
|
||||
const left = 'Hello beautiful world';
|
||||
const right = 'Hi world';
|
||||
|
||||
|
|
@ -68,9 +54,9 @@ const result = reconcile(parent, left, right);
|
|||
console.log(result.text); // "Hi beautiful world"
|
||||
```
|
||||
|
||||
See the [TypeScript guide][typescript-guide] for cursors, change provenance, and compact diffs, or the [demo website source][demo-source] for an editor integration.
|
||||
The [TypeScript guide][typescript-guide] shows how to track cursors, see where changes came from, and work with compact diffs. For an example of using the library in an editor, see the [demo website source][demo-source].
|
||||
|
||||
The Node.js and browser builds use the Rust library compiled to WebAssembly. For React Native, the package ships a pure-JavaScript build produced by [Binaryen's `wasm2js`][binaryen] via its `react-native` entry point, so it can run in Hermes without a `WebAssembly` global.
|
||||
The Node.js and browser builds use the Rust library compiled to WebAssembly. React Native uses a JavaScript build generated by [Binaryen's `wasm2js`][binaryen]. It's available through the package's `react-native` entry point and runs in Hermes without a `WebAssembly` global.
|
||||
|
||||
### Python
|
||||
|
||||
|
|
@ -81,8 +67,6 @@ uv add reconcile-text
|
|||
# or: pip install reconcile-text
|
||||
```
|
||||
|
||||
Then use it in your application:
|
||||
|
||||
```python
|
||||
from reconcile_text import reconcile
|
||||
|
||||
|
|
@ -94,65 +78,61 @@ result = reconcile(parent, left, right)
|
|||
print(result["text"]) # "Hi beautiful world"
|
||||
```
|
||||
|
||||
The Python package provides native bindings to the same Rust implementation. See the [merge-file example][python-merge-file] for a file-merging CLI, or the [Python guide][python-guide] for cursors, change provenance, and compact diffs.
|
||||
The Python package uses native bindings to the same Rust library. For merging files from the command line, see the [merge-file example][python-merge-file]. The [Python guide][python-guide] shows how to track cursors, see where changes came from, and work with compact diffs.
|
||||
|
||||
## Why merge snapshots?
|
||||
|
||||
Collaborative editing presents the challenge of merging conflicting changes when multiple users edit documents simultaneously or asynchronously whilst offline. A note might be edited in Obsidian on one device and Vim on another, with a file sync service carrying changes between them. By the time those changes meet, the syncer sees two files. It cannot ask either editor for the operations that produced them.
|
||||
Suppose you edit a note in Obsidian on one device and Vim on another while they're offline. When the devices reconnect, your file sync service has two edited copies to combine. It can read the files, but it can't ask either editor which operations produced them.
|
||||
|
||||
A common ancestor gives it enough context to compare both sets of edits. Git works with the same inputs, but usually leaves ambiguous conflicts for a person to resolve. That is a useful default for source code. For notes and prose, an automatic merge can be more useful, even if the wording occasionally needs tidying up.
|
||||
Keeping the original version gives it a common parent to compare both copies against. Git works with the same inputs and leaves conflicts it can't resolve for a person to handle. That's a useful default for source code. For notes and prose, an automatic merge can be more useful, even if the wording occasionally needs tidying up.
|
||||
|
||||
## How it works
|
||||
|
||||
`reconcile-text` starts off similarly to [`diff3`][diff3] but adds automated conflict resolution. Given a **parent** document and two modified versions (`left` and `right`), the following happens:
|
||||
Like [`diff3`][diff3], `reconcile-text` starts with a **parent** document and two edited versions, `left` and `right`. It then:
|
||||
|
||||
1. **Tokenisation** - Input texts are split into meaningful units (words, characters, etc.) for granular merging
|
||||
2. **Diff computation** - Myers' algorithm calculates differences between (parent ↔ left) and (parent ↔ right)
|
||||
3. **Diff optimisation** - Operations are reordered and consolidated to maximise chained changes
|
||||
4. **Operational Transformation** - Edits are woven together using OT principles, preserving all modifications and updating cursors
|
||||
1. **Splits the text into tokens**, such as words, characters, or lines.
|
||||
2. **Finds each side's changes** using Myers' diff algorithm to compare the parent with `left` and with `right`.
|
||||
3. **Cleans up the diffs** by reordering and grouping operations to keep related changes together.
|
||||
4. **Combines the edits** using operational transformation (OT) principles to account for changes from both sides and update cursor positions.
|
||||
|
||||
Whilst the primary goal of `reconcile-text` isn't to implement OT, it provides an elegant way to merge Myers' diff outputs. (For a dedicated Rust OT implementation, see [operational-transform-rs][operational-transform-rs].) The same could be achieved with CRDTs, which many libraries implement well for text (see [Loro][loro], [cola][cola], and [automerge][automerge]).
|
||||
|
||||
However, when only the end result of concurrent changes is observable, merge quality depends entirely on the quality of the underlying 2-way diffs. For instance, `move` operations cannot be supported because Myers' algorithm decomposes them into separate `insert` and `delete` operations, regardless of the merging algorithm used.
|
||||
|
||||
### Choosing a tokeniser
|
||||
|
||||
Tokenisation affects the result, especially when edits overlap. The built-in options are:
|
||||
The tokeniser decides which pieces of text stay together during a merge:
|
||||
|
||||
- **Word**: keeps words together; the default in JavaScript/TypeScript and Python.
|
||||
- **Character**: merges individual Unicode code points for finer-grained changes.
|
||||
- **Line**: treats each line as a unit.
|
||||
- **Markdown**: builds on word tokenisation, preserving newlines and attaching block prefixes such as list markers to their first word.
|
||||
- **Word** keeps words together. This is the default in JavaScript/TypeScript and Python.
|
||||
- **Character** works with individual Unicode code points, so edits can be combined within a word.
|
||||
- **Line** treats each line as a unit.
|
||||
- **Markdown** builds on word tokenisation. It preserves newlines and keeps block prefixes, such as list markers, attached to the first word.
|
||||
|
||||
Rust callers choose a tokeniser explicitly and can also supply a custom function. The merge keeps tokens intact, but the Markdown tokeniser does not validate the resulting document's syntax. Try different tokenisers in the [demo][demo] to see how they handle your text.
|
||||
In Rust, you choose a tokeniser explicitly and can supply your own function. The merge keeps tokens intact, though the Markdown tokeniser doesn't check whether the merged document is valid Markdown. Try the options in the [demo][demo] to see how they handle your text.
|
||||
|
||||
## Comparison with other approaches
|
||||
|
||||
### diff3 and Git
|
||||
|
||||
`diff3` and Git's default text merge leave conflict markers where changes cannot be combined automatically. Git also has a [`union` merge driver][git-merge-drivers] that retains lines from both sides of a conflict. Because it works with whole lines, two edits to different words in a sentence can leave two versions of the sentence. Word tokenisation lets `reconcile-text` combine those edits within the line.
|
||||
`diff3` and Git's default text merge leave conflict markers where they can't combine changes automatically. Git also has a [`union` merge driver][git-merge-drivers] that keeps lines from both sides of a conflict. Because it works with whole lines, edits to different words in the same sentence can leave you with two versions of that sentence. With word tokenisation, `reconcile-text` can combine those edits within the line.
|
||||
|
||||
### diff-match-patch
|
||||
|
||||
[diff-match-patch][diff-match-patch] computes two-way diffs and applies patches using fuzzy matching. It is useful when you need to apply changes to text whose context may have shifted. A patch can fail, so callers need to inspect its success flags.
|
||||
[diff-match-patch][diff-match-patch] compares two texts and applies patches using fuzzy matching. This is useful when the text around a change may have shifted. Patches can fail, so you need to check the success flags returned when applying them.
|
||||
|
||||
`reconcile-text` uses the common parent to derive both sides' changes before combining them. The [comparison example][comparison-example] shows how the two approaches handle overlapping replacements and a sentence inserted into a rewritten paragraph.
|
||||
`reconcile-text` compares each edited version with the common parent before combining the changes. The [comparison example][comparison-example] shows how both approaches handle overlapping replacements and a sentence inserted into a rewritten paragraph.
|
||||
|
||||
### CRDTs and OT
|
||||
|
||||
Libraries based on conflict-free replicated data types (CRDTs), such as [Yjs][yjs], [Automerge][automerge], and [Loro][loro], maintain document structures and metadata that let replicas merge updates. They fit applications that can persist that state and integrate edits into it. Snapshot changes can also be translated into CRDT updates; using a CRDT does not require recording every keystroke.
|
||||
Libraries such as [Yjs][yjs], [Automerge][automerge], [Loro][loro], and [cola][cola] use conflict-free replicated data types (CRDTs) to merge updates across replicas. They keep document structures and metadata between edits, so your application needs to store and work with that state. You can also turn differences between snapshots into CRDT updates; using a CRDT doesn't require recording every keystroke.
|
||||
|
||||
OT libraries such as [ot.js][ot-js] work with operations and a protocol for coordinating them. `reconcile-text` uses OT ideas internally, deriving the operations from each pair of snapshots at merge time.
|
||||
With OT, applications exchange editing operations and use a protocol to coordinate them. Libraries such as [ot.js][ot-js] and Rust's [operational-transform-rs][operational-transform-rs] provide tools for transforming those operations. `reconcile-text` uses OT ideas internally, deriving its operations from the snapshots when you call it.
|
||||
|
||||
The practical distinction is what your application keeps between merges. `reconcile-text` accepts three texts and merges two forks at a time. A collaborative editing system built around a CRDT or OT also manages the state and protocol needed to coordinate replicas over time.
|
||||
The main difference is what your application keeps between merges. `reconcile-text` takes a parent and two edited copies, and merges those two copies in one call. A collaborative editing system built around CRDTs or OT also manages the state and coordination needed to keep replicas in sync over time.
|
||||
|
||||
## Development
|
||||
|
||||
Contributions are welcome!
|
||||
|
||||
### Environment
|
||||
|
||||
#### Python setup
|
||||
### Python setup
|
||||
|
||||
Install [uv][uv] and build the extension for development:
|
||||
|
||||
|
|
@ -161,7 +141,7 @@ cd reconcile-python
|
|||
uv run maturin develop
|
||||
```
|
||||
|
||||
#### Node.js setup
|
||||
### Node.js setup
|
||||
|
||||
1. Install [nvm][nvm]:
|
||||
```sh
|
||||
|
|
@ -171,25 +151,26 @@ uv run maturin develop
|
|||
```sh
|
||||
nvm install 22 && nvm use 22
|
||||
```
|
||||
3. Optionally, set as default:
|
||||
3. Optionally, make Node 22 your default:
|
||||
```sh
|
||||
nvm alias default 22
|
||||
```
|
||||
|
||||
#### Rust toolchain
|
||||
### Rust toolchain
|
||||
|
||||
Install [rustup][rustup]:
|
||||
```bash
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
||||
```
|
||||
|
||||
```sh
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
||||
```
|
||||
|
||||
### Scripts
|
||||
|
||||
- **Run tests**: `scripts/test.sh`
|
||||
- **Check lint and formatting**: `scripts/lint.sh`
|
||||
- **Develop demo website**: `scripts/dev-website.sh`
|
||||
- **Build demo website**: `scripts/build-website.sh`
|
||||
- **Publish new version**: `scripts/bump-version.sh patch`
|
||||
- **Run the demo locally**: `scripts/dev-website.sh`
|
||||
- **Build the demo**: `scripts/build-website.sh`
|
||||
- **Publish a new version**: `scripts/bump-version.sh patch`
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue