This commit is contained in:
parent
a2fbba682e
commit
b4259b767c
13 changed files with 608 additions and 24 deletions
38
README.md
38
README.md
|
|
@ -1,6 +1,6 @@
|
|||
# `reconcile-text`: conflict-free 3-way text merging
|
||||
|
||||
A Rust, TypeScript, and Python library for merging conflicting text edits without manual intervention. Unlike traditional 3-way merge tools that produce conflict markers, `reconcile-text` automatically resolves conflicts by applying both sets of changes (while updating cursor positions) using an algorithm inspired by Operational Transformation.
|
||||
A Rust, TypeScript, and Python library for merging conflicting text edits without manual intervention. Unlike traditional 3-way merge tools that produce conflict markers, `reconcile-text` automatically resolves conflicts using an algorithm inspired by Operational Transformation. It combines edits where possible and prioritizes stable replay while updating cursor positions.
|
||||
|
||||
## Try it
|
||||
|
||||
|
|
@ -132,12 +132,46 @@ Differential sync is implemented by [universal-sync][12], and it requires a merg
|
|||
1. **Tokenisation** - Input texts are split into meaningful units (words, characters, etc.) for granular merging
|
||||
2. **Diff computation** - Myers' algorithm calculates differences between (parent ↔ left) and (parent ↔ right)
|
||||
3. **Diff optimisation** - Operations are reordered and consolidated to maximise chained changes
|
||||
4. **Operational Transformation** - Edits are woven together using OT principles, preserving all modifications and updating cursors
|
||||
4. **Operational Transformation** - Edits are woven together using OT principles, combining changes and updating cursors
|
||||
5. **Replay verification** - Combined text is accepted only when it absorbs both inputs on replay; ambiguous cases keep one input deterministically
|
||||
|
||||
Whilst the primary goal of `reconcile-text` isn't to implement OT, it provides an elegant way to merge Myers' diff outputs. (For a dedicated Rust OT implementation, see [operational-transform-rs][13].) The same could be achieved with CRDTs, which many libraries implement well for text (see [Loro][14], [cola][15], and [automerge][16]).
|
||||
|
||||
However, when only the end result of concurrent changes is observable, merge quality depends entirely on the quality of the underlying 2-way diffs. For instance, `move` operations cannot be supported because Myers' algorithm decomposes them into separate `insert` and `delete` operations, regardless of the merging algorithm used.
|
||||
|
||||
### Replay stability
|
||||
|
||||
Reapplying either unchanged input against the same base preserves the merged text:
|
||||
|
||||
```javascript
|
||||
const merged = reconcile('a', 'b', 'x').text; // 'bx'
|
||||
reconcile('a', merged, 'x').text; // 'bx'
|
||||
reconcile('a', merged, 'b').text; // 'bx'
|
||||
```
|
||||
|
||||
This holds for every built-in tokenizer and deterministic, lossless custom
|
||||
tokenizers whose original token text concatenates back to the input.
|
||||
The guarantee applies to text; cursors from both supplied inputs are mapped into
|
||||
the result on each call.
|
||||
|
||||
The merge treats insertions already included in the other side's inserted text
|
||||
as incorporated, including character subsequences inside joined words. It checks
|
||||
that a tentative combined result absorbs both inputs. If it does not, the merge
|
||||
keeps one input deterministically: more inserted characters take precedence,
|
||||
then more deleted characters, then lexicographically greater text.
|
||||
|
||||
This gives retry stability precedence over preserving every concurrent edit.
|
||||
For example, changing `a` to `abc` and `abcd` now produces `abcd`. Plain strings
|
||||
cannot distinguish a replay from every independent edit containing the same
|
||||
text, so some concurrent edits can be superseded. The fallback selects an input
|
||||
without iterating until convergence; concurrent merges perform at most two
|
||||
additional candidate merges to verify replay.
|
||||
|
||||
In Rust, use `reconcile` or `EditedText::merge_with_tokenizer` for this guarantee.
|
||||
The lower-level `EditedText::merge` combines operations without the snapshot
|
||||
replay check. Continue advancing the sync base after a successful application;
|
||||
this guarantee covers replay of the unchanged merge result.
|
||||
|
||||
## Comparison with other approaches
|
||||
|
||||
### Traditional 3-way merge (diff3, Git)
|
||||
|
|
|
|||
|
|
@ -143,6 +143,9 @@ export interface ReconcileApi {
|
|||
* This is the primary function for 3-way text merging. Unlike traditional merge tools
|
||||
* that produce conflict markers, this function automatically resolves conflicts by
|
||||
* applying both sets of changes where possible.
|
||||
* Replaying either unchanged input against the same base preserves the merged
|
||||
* text. Replay stability can supersede ambiguous concurrent edits; the guarantee
|
||||
* applies to text, not the combined cursor list.
|
||||
*
|
||||
* @param original - The original/base version of the text that both sides diverged from
|
||||
* @param left - The left version of the text (either string or TextWithCursors with cursor positions)
|
||||
|
|
|
|||
|
|
@ -73,6 +73,40 @@ describe.each(backends)('reconcile [$name]', ({ api }) => {
|
|||
expect(result.history.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it.each(['a', ''])(
|
||||
'replaying conflicting replacements with base %p keeps both edits',
|
||||
(base) => {
|
||||
for (const tokenizer of ['Character', 'Line', 'Markdown', 'Word'] as const) {
|
||||
const merged = reconcile(base, 'b', 'x', tokenizer);
|
||||
expect(merged.text).toBe('bx');
|
||||
expect(reconcile(base, merged, 'x', tokenizer)).toEqual(merged);
|
||||
expect(reconcile(base, merged, 'b', tokenizer)).toEqual(merged);
|
||||
expect(reconcileWithHistory(base, merged, 'x', tokenizer).text).toBe('bx');
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
it('preserves intentional repeats and Unicode cursor positions on replay', () => {
|
||||
const repeated = reconcile('a', 'bb', 'x');
|
||||
expect(repeated.text).toBe('bbx');
|
||||
expect(reconcile('a', repeated, 'x')).toEqual(repeated);
|
||||
const local = {
|
||||
text: '🦊x',
|
||||
cursors: [
|
||||
{ id: 1, position: 1 },
|
||||
{ id: 2, position: 2 },
|
||||
],
|
||||
};
|
||||
const result = reconcile('a', local, {
|
||||
text: 'x',
|
||||
cursors: [{ id: 3, position: 1 }],
|
||||
});
|
||||
expect(result.text).toBe(local.text);
|
||||
expect(result.cursors).toEqual(
|
||||
expect.arrayContaining([...local.cursors, { id: 3, position: 2 }])
|
||||
);
|
||||
});
|
||||
|
||||
it('undiff accepts bigint entries (per the Array<number | bigint | string> type)', () => {
|
||||
const original = 'Hello world';
|
||||
const changed = 'Hello cruel world';
|
||||
|
|
|
|||
|
|
@ -27,6 +27,19 @@ class TestReconcile:
|
|||
result = reconcile(parent, left, right)
|
||||
assert result["text"] == "With reconcile, merging documents is easy!"
|
||||
|
||||
@pytest.mark.parametrize("parent", ["", "a"])
|
||||
def test_replaying_conflicting_replacements(self, parent: str) -> None:
|
||||
merged = reconcile(parent, "b", "x")
|
||||
assert merged["text"] == "bx"
|
||||
assert reconcile(parent, merged, "x") == merged
|
||||
assert reconcile(parent, merged, "b") == merged
|
||||
assert reconcile_with_history(parent, merged, "x")["text"] == "bx"
|
||||
|
||||
def test_replaying_preserves_repeated_text(self) -> None:
|
||||
merged = reconcile("a", "bb", "x")
|
||||
assert merged["text"] == "bbx"
|
||||
assert reconcile("a", merged, "x") == merged
|
||||
|
||||
def test_with_cursors(self) -> None:
|
||||
result = reconcile(
|
||||
"Hello",
|
||||
|
|
|
|||
|
|
@ -2,8 +2,9 @@
|
|||
//!
|
||||
//! A library for merging conflicting text edits without manual intervention.
|
||||
//! Unlike traditional 3-way merge tools that produce conflict markers,
|
||||
//! reconcile-text automatically resolves conflicts by applying both sets of
|
||||
//! changes (while updating cursor positions) using an algorithm inspired by
|
||||
//! reconcile-text automatically resolves conflicts by combining edits where
|
||||
//! possible, prioritizing stable replay, and updating cursor positions with
|
||||
//! an algorithm inspired by
|
||||
//! Operational Transformation.
|
||||
//!
|
||||
//! ✨ **[Try the interactive demo](https://schmelczer.dev/reconcile)** to see it in action.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
mod diff_error;
|
||||
mod edited_text;
|
||||
mod operation;
|
||||
mod replay;
|
||||
mod utils;
|
||||
use std::fmt::Debug;
|
||||
|
||||
|
|
@ -10,22 +11,22 @@ pub use operation::Operation;
|
|||
|
||||
use crate::{Tokenizer, types::text_with_cursors::TextWithCursors};
|
||||
|
||||
/// Given an `original` document and two concurrent edits to it,
|
||||
/// return a document containing all changes from both `left`
|
||||
/// and `right`.
|
||||
/// Merge two concurrent snapshots, prioritizing stability when either input
|
||||
/// is replayed against the same `original` document.
|
||||
///
|
||||
/// If both updated texts are identical, the resulting text is exactly that
|
||||
/// text, regardless of the original document.
|
||||
///
|
||||
/// If a span has been inserted in either the `left` or `right`
|
||||
/// versions, it will be present in the return value. If both sides
|
||||
/// insert the same span with a common prefix, that prefix will only
|
||||
/// be present once in the output.
|
||||
/// With a deterministic, lossless tokenizer, if `merged = reconcile(original, left,
|
||||
/// right).apply()`, reconciling `merged` with either input again produces the
|
||||
/// same text. This guarantee concerns text, not the combined cursor list.
|
||||
///
|
||||
/// When both sides delete the same span, it will be deleted in the
|
||||
/// return value. If one side deletes a span and the other side inserts
|
||||
/// into that span, the inserted text will be present in the return
|
||||
/// value.
|
||||
/// Already incorporated insertions are recognized as character subsequences
|
||||
/// of the other side's inserted text, including inside joined words. A combined
|
||||
/// result is kept only when it absorbs both inputs on replay. Otherwise one
|
||||
/// snapshot is selected, preferring more inserted characters, then more deleted
|
||||
/// characters, then lexicographically greater text. Some independent concurrent
|
||||
/// edits can therefore be superseded to keep retries stable.
|
||||
///
|
||||
/// Supports UTF-8. Arguments are tokenized using the provided `tokenizer`.
|
||||
///
|
||||
|
|
@ -49,10 +50,7 @@ pub fn reconcile<'a, T>(
|
|||
where
|
||||
T: PartialEq + Clone + Debug,
|
||||
{
|
||||
let left_operations = EditedText::from_strings_with_tokenizer(original, left, tokenizer);
|
||||
let right_operations = EditedText::from_strings_with_tokenizer(original, right, tokenizer);
|
||||
|
||||
left_operations.merge(right_operations)
|
||||
replay::merge_stable(original, left, right, tokenizer)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
|
|||
|
|
@ -95,10 +95,58 @@ where
|
|||
}
|
||||
}
|
||||
|
||||
pub(crate) fn inserted_text(&self) -> String {
|
||||
let mut result = String::new();
|
||||
for operation in &self.operations {
|
||||
if let Operation::Insert { text, .. } = operation {
|
||||
for token in text {
|
||||
result.push_str(token.original());
|
||||
}
|
||||
}
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
pub(crate) fn edit_counts(&self) -> (usize, usize) {
|
||||
self.operations
|
||||
.iter()
|
||||
.fold((0, 0), |(inserted, deleted), operation| match operation {
|
||||
Operation::Insert { .. } => (inserted + operation.len(), deleted),
|
||||
Operation::Delete {
|
||||
deleted_character_count,
|
||||
..
|
||||
} => (inserted, deleted + deleted_character_count),
|
||||
Operation::Equal { .. } => (inserted, deleted),
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn with_cursors_and_side(
|
||||
mut self,
|
||||
mut cursors: Vec<CursorPosition>,
|
||||
side: Side,
|
||||
) -> Self {
|
||||
cursors.sort_by_key(|cursor| cursor.char_index);
|
||||
self.cursors = cursors;
|
||||
self.operation_sides.fill(side);
|
||||
self
|
||||
}
|
||||
|
||||
pub(crate) fn swap_sides(mut self) -> Self {
|
||||
for side in &mut self.operation_sides {
|
||||
*side = match side {
|
||||
Side::Left => Side::Right,
|
||||
Side::Right => Side::Left,
|
||||
};
|
||||
}
|
||||
self
|
||||
}
|
||||
|
||||
/// Merge two `EditedText` instances. The two instances must be derived
|
||||
/// from the same original text. The operations are merged using the
|
||||
/// principles of Operational Transformation. The cursors are updated
|
||||
/// accordingly to reflect the changes made by the merged operations.
|
||||
/// This operation-level API does not guarantee snapshot replay stability;
|
||||
/// use [`Self::merge_with_tokenizer`] or [`crate::reconcile`] for that contract.
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
|
|
@ -254,6 +302,23 @@ where
|
|||
)
|
||||
}
|
||||
|
||||
/// Merge snapshots with the same replay guarantees as [`crate::reconcile`].
|
||||
/// The tokenizer must be deterministic, lossless, and match the tokenizer used
|
||||
/// to create these edits. Operations are recalculated from their snapshots.
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics in debug builds if the original texts differ, or if a cursor index
|
||||
/// overflows an `isize` during transformation.
|
||||
#[must_use]
|
||||
pub fn merge_with_tokenizer(&self, other: &Self, tokenizer: &Tokenizer<T>) -> Self {
|
||||
debug_assert_eq!(
|
||||
self.text, other.text,
|
||||
"Edits must share the same original text"
|
||||
);
|
||||
super::reconcile(self.text, &self.apply(), &other.apply(), tokenizer)
|
||||
}
|
||||
|
||||
/// Apply the operations to the text and return the resulting text
|
||||
#[must_use]
|
||||
pub fn apply(&self) -> TextWithCursors {
|
||||
|
|
|
|||
172
src/operation_transformation/replay.rs
Normal file
172
src/operation_transformation/replay.rs
Normal file
|
|
@ -0,0 +1,172 @@
|
|||
use std::fmt::Debug;
|
||||
|
||||
use super::EditedText;
|
||||
use crate::{BuiltinTokenizer, Side, TextWithCursors, Tokenizer};
|
||||
|
||||
pub(super) fn merge_stable<'a, T>(
|
||||
original: &'a str,
|
||||
left: &TextWithCursors,
|
||||
right: &TextWithCursors,
|
||||
tokenizer: &Tokenizer<T>,
|
||||
) -> EditedText<'a, T>
|
||||
where
|
||||
T: PartialEq + Clone + Debug,
|
||||
{
|
||||
// A combined result is accepted only if the candidate merge absorbs both
|
||||
// inputs. On replay the candidate is then the same result, and its checks
|
||||
// are unchanged or compare identical snapshots. If this fails, selecting
|
||||
// one input is also stable: replay sees the original pair or identical inputs.
|
||||
let candidate = merge_candidate(original, left, right, tokenizer);
|
||||
if left.text_ref() == right.text_ref()
|
||||
|| left.text_ref() == original
|
||||
|| right.text_ref() == original
|
||||
{
|
||||
return candidate;
|
||||
}
|
||||
let merged = candidate.apply().text();
|
||||
let without_cursors = merged.clone().into();
|
||||
if [left, right].iter().all(|input| {
|
||||
merge_candidate(
|
||||
original,
|
||||
&without_cursors,
|
||||
&input.text_ref().into(),
|
||||
tokenizer,
|
||||
)
|
||||
.apply()
|
||||
.text()
|
||||
== merged
|
||||
}) {
|
||||
return candidate;
|
||||
}
|
||||
|
||||
let left_operations = EditedText::from_strings_with_tokenizer(original, left, tokenizer);
|
||||
let right_operations = EditedText::from_strings_with_tokenizer(original, right, tokenizer);
|
||||
if version_priority(&left_operations, left) >= version_priority(&right_operations, right) {
|
||||
keep_version(left_operations, right_operations, left, right, Side::Left)
|
||||
} else {
|
||||
keep_version(right_operations, left_operations, right, left, Side::Right)
|
||||
}
|
||||
}
|
||||
|
||||
fn is_subsequence(part: &str, whole: &str) -> bool {
|
||||
let mut chars = whole.chars();
|
||||
part.chars()
|
||||
.all(|ch| chars.by_ref().any(|candidate| candidate == ch))
|
||||
}
|
||||
|
||||
fn version_priority<'b, T>(
|
||||
operations: &EditedText<'_, T>,
|
||||
input: &'b TextWithCursors,
|
||||
) -> (usize, usize, &'b str)
|
||||
where
|
||||
T: PartialEq + Clone + Debug,
|
||||
{
|
||||
let (inserted, deleted) = operations.edit_counts();
|
||||
(inserted, deleted, input.text_ref())
|
||||
}
|
||||
|
||||
fn merge_candidate<'a, T>(
|
||||
original: &'a str,
|
||||
left: &TextWithCursors,
|
||||
right: &TextWithCursors,
|
||||
tokenizer: &Tokenizer<T>,
|
||||
) -> EditedText<'a, T>
|
||||
where
|
||||
T: PartialEq + Clone + Debug,
|
||||
{
|
||||
let left_operations = EditedText::from_strings_with_tokenizer(original, left, tokenizer);
|
||||
let right_operations = EditedText::from_strings_with_tokenizer(original, right, tokenizer);
|
||||
let left_text = left.text_ref();
|
||||
let right_text = right.text_ref();
|
||||
if left_text == right_text || left_text == original || right_text == original {
|
||||
return merge_ordered(left_operations, right_operations, left_text, right_text);
|
||||
}
|
||||
|
||||
let left_inserted = left_operations.inserted_text();
|
||||
let right_inserted = right_operations.inserted_text();
|
||||
// Treat insertions already present in the other side's edits as incorporated,
|
||||
// including inside joined words or across separate inserted spans. This
|
||||
// deliberately gives replay precedence over ambiguous independent edits.
|
||||
let left_contains_right =
|
||||
!right_inserted.is_empty() && is_subsequence(&right_inserted, &left_inserted);
|
||||
let right_contains_left =
|
||||
!left_inserted.is_empty() && is_subsequence(&left_inserted, &right_inserted);
|
||||
match (left_contains_right, right_contains_left) {
|
||||
(true, false) => keep_version(left_operations, right_operations, left, right, Side::Left),
|
||||
(false, true) => keep_version(right_operations, left_operations, right, left, Side::Right),
|
||||
(true, true) => {
|
||||
if version_priority(&left_operations, left)
|
||||
>= version_priority(&right_operations, right)
|
||||
{
|
||||
keep_version(left_operations, right_operations, left, right, Side::Left)
|
||||
} else {
|
||||
keep_version(right_operations, left_operations, right, left, Side::Right)
|
||||
}
|
||||
}
|
||||
(false, false) => merge_ordered(left_operations, right_operations, left_text, right_text),
|
||||
}
|
||||
}
|
||||
|
||||
fn merge_ordered<'a, T>(
|
||||
left: EditedText<'a, T>,
|
||||
right: EditedText<'a, T>,
|
||||
left_text: &str,
|
||||
right_text: &str,
|
||||
) -> EditedText<'a, T>
|
||||
where
|
||||
T: PartialEq + Clone + Debug,
|
||||
{
|
||||
// Canonical input order also stabilizes tokenizers with normalized or
|
||||
// non-joinable tokens. Restore provenance to the caller's original sides.
|
||||
if left_text <= right_text {
|
||||
left.merge(right)
|
||||
} else {
|
||||
right.merge(left).swap_sides()
|
||||
}
|
||||
}
|
||||
|
||||
fn keep_version<'a, T>(
|
||||
operations: EditedText<'a, T>,
|
||||
other_operations: EditedText<'a, T>,
|
||||
kept: &TextWithCursors,
|
||||
other: &TextWithCursors,
|
||||
side: Side,
|
||||
) -> EditedText<'a, T>
|
||||
where
|
||||
T: PartialEq + Clone + Debug,
|
||||
{
|
||||
let kept_text = kept.text_ref();
|
||||
let traditional = merge_ordered(
|
||||
operations.clone(),
|
||||
other_operations,
|
||||
kept_text,
|
||||
other.text_ref(),
|
||||
);
|
||||
// Preserve the existing cursor mapping and provenance when the operation
|
||||
// merge already produces the selected text. Character projection is needed
|
||||
// only when replay suppression actually changes the operation result.
|
||||
if traditional.apply().text_ref() == kept_text {
|
||||
return match side {
|
||||
Side::Left => traditional,
|
||||
Side::Right => traditional.swap_sides(),
|
||||
};
|
||||
}
|
||||
let mut cursors = kept.cursors();
|
||||
if !other.cursors().is_empty() {
|
||||
let other_text = other.text_ref();
|
||||
// Character alignment maps cursors inside joined words and uses the
|
||||
// same character offsets as the public cursor API, including Unicode.
|
||||
let original = EditedText::from_strings_with_tokenizer(
|
||||
other_text,
|
||||
other,
|
||||
&*BuiltinTokenizer::Character,
|
||||
);
|
||||
let updated = EditedText::from_strings_with_tokenizer(
|
||||
other_text,
|
||||
&kept_text.into(),
|
||||
&*BuiltinTokenizer::Character,
|
||||
);
|
||||
cursors.extend(original.merge(updated).apply().cursors());
|
||||
}
|
||||
operations.with_cursors_and_side(cursors, side)
|
||||
}
|
||||
|
|
@ -21,4 +21,4 @@ expected: a b c d
|
|||
parent: a
|
||||
left: abc
|
||||
right: abcd
|
||||
expected: abcabcd
|
||||
expected: abcd
|
||||
|
|
|
|||
|
|
@ -16,4 +16,4 @@ expected: original_1 edit_1| original_3|
|
|||
parent: original_1 original_2 original_3
|
||||
left: original_1 edit_1| original_3
|
||||
right: original_1 conflicting_edit_1| original_3
|
||||
expected: original_1 conflicting_edit_1| edit_1| original_3
|
||||
expected: original_1 conflicting_edit_1|| original_3
|
||||
|
|
|
|||
|
|
@ -85,7 +85,7 @@ expected: Please submit your |completed |assignment |online |by Friday
|
|||
parent: "a b "
|
||||
left: "c d "
|
||||
right: "a b c d "
|
||||
expected: "c d c d "
|
||||
expected: "a b c d "
|
||||
|
||||
---
|
||||
parent: a b c d e
|
||||
|
|
|
|||
|
|
@ -1,4 +1,6 @@
|
|||
use reconcile_text::{BuiltinTokenizer, Token, reconcile};
|
||||
use reconcile_text::{
|
||||
BuiltinTokenizer, CursorPosition, EditedText, TextWithCursors, Token, reconcile,
|
||||
};
|
||||
use test_case::{test_case, test_matrix};
|
||||
|
||||
#[test_matrix([
|
||||
|
|
@ -79,3 +81,265 @@ fn identical_edits_preserve_original_spelling(parent: &str, updated: &str) {
|
|||
updated,
|
||||
);
|
||||
}
|
||||
|
||||
#[test_matrix([
|
||||
BuiltinTokenizer::Character,
|
||||
BuiltinTokenizer::Line,
|
||||
BuiltinTokenizer::Markdown,
|
||||
BuiltinTokenizer::Word
|
||||
])]
|
||||
fn replay_short_edits_preserves_merged_text(tokenizer: BuiltinTokenizer) {
|
||||
let mut texts = vec![String::new()];
|
||||
let mut level = texts.clone();
|
||||
for _ in 0..2 {
|
||||
level = level
|
||||
.iter()
|
||||
.flat_map(|text| ['a', 'b', ' ', '\n'].map(|ch| format!("{text}{ch}")))
|
||||
.collect();
|
||||
texts.extend(level.iter().cloned());
|
||||
}
|
||||
|
||||
for parent in &texts {
|
||||
for left in &texts {
|
||||
for right in &texts {
|
||||
let merged = reconcile(parent, &left.into(), &right.into(), &*tokenizer).apply();
|
||||
assert_eq!(
|
||||
reconcile(parent, &right.into(), &left.into(), &*tokenizer)
|
||||
.apply()
|
||||
.text(),
|
||||
merged.text(),
|
||||
"swapped inputs: tokenizer={tokenizer:?}, parent={parent:?}, left={left:?}, right={right:?}",
|
||||
);
|
||||
for replayed in [left, right] {
|
||||
let actual = reconcile(parent, &merged, &replayed.into(), &*tokenizer).apply();
|
||||
assert_eq!(
|
||||
actual.text(),
|
||||
merged.text(),
|
||||
"tokenizer={tokenizer:?}, parent={parent:?}, left={left:?}, right={right:?}, replayed={replayed:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test_matrix(["", "a"], [
|
||||
BuiltinTokenizer::Character,
|
||||
BuiltinTokenizer::Line,
|
||||
BuiltinTokenizer::Markdown,
|
||||
BuiltinTokenizer::Word
|
||||
])]
|
||||
fn replay_conflicting_replacements_keeps_both_edits(parent: &str, tokenizer: BuiltinTokenizer) {
|
||||
let left = TextWithCursors::new("b".to_owned(), vec![CursorPosition::new(1, 1)]);
|
||||
let right = TextWithCursors::new("x".to_owned(), vec![CursorPosition::new(2, 1)]);
|
||||
let merged = reconcile(parent, &left, &right, &*tokenizer).apply();
|
||||
assert_eq!(merged.text(), "bx");
|
||||
assert_eq!(
|
||||
merged.cursors(),
|
||||
vec![CursorPosition::new(1, 1), CursorPosition::new(2, 2)]
|
||||
);
|
||||
for input in ["b", "x"] {
|
||||
let replayed = reconcile(parent, &merged, &input.into(), &*tokenizer).apply();
|
||||
assert_eq!(replayed, merged);
|
||||
}
|
||||
let left_operations = EditedText::from_strings_with_tokenizer(parent, &left, &*tokenizer);
|
||||
let right_operations = EditedText::from_strings_with_tokenizer(parent, &right, &*tokenizer);
|
||||
assert_eq!(
|
||||
left_operations
|
||||
.merge_with_tokenizer(&right_operations, &*tokenizer)
|
||||
.apply(),
|
||||
merged
|
||||
);
|
||||
}
|
||||
|
||||
#[test_matrix([
|
||||
BuiltinTokenizer::Character,
|
||||
BuiltinTokenizer::Line,
|
||||
BuiltinTokenizer::Markdown,
|
||||
BuiltinTokenizer::Word
|
||||
])]
|
||||
fn replay_preserves_repeated_text_and_unicode_cursors(tokenizer: BuiltinTokenizer) {
|
||||
let merged = reconcile("a", &"bb".into(), &"x".into(), &*tokenizer).apply();
|
||||
assert_eq!(merged.text(), "bbx");
|
||||
assert_eq!(
|
||||
reconcile("a", &merged, &"x".into(), &*tokenizer).apply(),
|
||||
merged
|
||||
);
|
||||
|
||||
let left = TextWithCursors::new(
|
||||
"🦊x".to_owned(),
|
||||
vec![CursorPosition::new(1, 1), CursorPosition::new(2, 2)],
|
||||
);
|
||||
let right = TextWithCursors::new("x".to_owned(), vec![CursorPosition::new(3, 1)]);
|
||||
let edited = reconcile("a", &left, &right, &*tokenizer);
|
||||
let result = edited.apply();
|
||||
assert_eq!(result.text(), "🦊x");
|
||||
for (id, position) in [(1, 1), (2, 2), (3, 2)] {
|
||||
assert!(
|
||||
result
|
||||
.cursors()
|
||||
.contains(&CursorPosition::new(id, position))
|
||||
);
|
||||
}
|
||||
let history_text: String = edited
|
||||
.apply_with_history()
|
||||
.iter()
|
||||
.filter(|span| {
|
||||
!matches!(
|
||||
span.history(),
|
||||
reconcile_text::History::RemovedFromLeft
|
||||
| reconcile_text::History::RemovedFromRight
|
||||
)
|
||||
})
|
||||
.map(reconcile_text::SpanWithHistory::text)
|
||||
.collect();
|
||||
assert_eq!(history_text, result.text());
|
||||
assert_eq!(edited.apply_with_all().0, result);
|
||||
}
|
||||
|
||||
#[test_matrix([
|
||||
BuiltinTokenizer::Character,
|
||||
BuiltinTokenizer::Line,
|
||||
BuiltinTokenizer::Markdown,
|
||||
BuiltinTokenizer::Word
|
||||
])]
|
||||
fn replay_longer_mixed_edits_is_stable(tokenizer: BuiltinTokenizer) {
|
||||
let mut seed = 0x12ab_34cd_56ef_7890u64;
|
||||
let mut next = || {
|
||||
seed ^= seed << 13;
|
||||
seed ^= seed >> 7;
|
||||
seed ^= seed << 17;
|
||||
seed
|
||||
};
|
||||
let alphabet = ['a', 'b', 'c', ' ', '\n', '\r', 'é', '🦊'];
|
||||
for case in 0..2000 {
|
||||
let mut text = || {
|
||||
(0..next() % 16)
|
||||
.map(|_| alphabet[usize::try_from(next() % 8).unwrap()])
|
||||
.collect::<String>()
|
||||
};
|
||||
let parent = text();
|
||||
let left = text();
|
||||
let right = text();
|
||||
let merged = reconcile(
|
||||
&parent,
|
||||
&left.clone().into(),
|
||||
&right.clone().into(),
|
||||
&*tokenizer,
|
||||
)
|
||||
.apply();
|
||||
if case < 200 {
|
||||
let left_with_cursors = TextWithCursors::new(
|
||||
left.clone(),
|
||||
(0..=left.chars().count())
|
||||
.map(|position| CursorPosition::new(position, position))
|
||||
.collect(),
|
||||
);
|
||||
let right_with_cursors = TextWithCursors::new(
|
||||
right.clone(),
|
||||
(0..=right.chars().count())
|
||||
.map(|position| CursorPosition::new(100 + position, position))
|
||||
.collect(),
|
||||
);
|
||||
let result = reconcile(
|
||||
&parent,
|
||||
&left_with_cursors,
|
||||
&right_with_cursors,
|
||||
&*tokenizer,
|
||||
)
|
||||
.apply();
|
||||
assert_eq!(result.text(), merged.text());
|
||||
assert_eq!(
|
||||
result.cursors().len(),
|
||||
left_with_cursors.cursors().len() + right_with_cursors.cursors().len()
|
||||
);
|
||||
assert!(
|
||||
result
|
||||
.cursors()
|
||||
.iter()
|
||||
.all(|cursor| cursor.char_index() <= result.text().chars().count())
|
||||
);
|
||||
for cursor in left_with_cursors
|
||||
.cursors()
|
||||
.into_iter()
|
||||
.chain(right_with_cursors.cursors())
|
||||
{
|
||||
assert!(
|
||||
result
|
||||
.cursors()
|
||||
.iter()
|
||||
.any(|updated| updated.id() == cursor.id())
|
||||
);
|
||||
}
|
||||
}
|
||||
for input in [&left, &right] {
|
||||
assert_eq!(
|
||||
reconcile(&parent, &merged, &input.into(), &*tokenizer)
|
||||
.apply()
|
||||
.text(),
|
||||
merged.text(),
|
||||
"tokenizer={tokenizer:?}, parent={parent:?}, left={left:?}, right={right:?}, input={input:?}"
|
||||
);
|
||||
}
|
||||
assert_eq!(
|
||||
reconcile(&parent, &right.into(), &left.into(), &*tokenizer)
|
||||
.apply()
|
||||
.text(),
|
||||
merged.text()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_with_normalized_tokens_preserves_spelling() {
|
||||
let tokenizer = |text: &str| {
|
||||
text.chars()
|
||||
.map(|ch| Token::new(ch.to_ascii_lowercase(), ch.to_string(), false, false))
|
||||
.collect()
|
||||
};
|
||||
let texts = ["", "A", "a", "AB", "ba", "Aa", "a A", " A", "\nA"];
|
||||
for parent in texts {
|
||||
for left in texts {
|
||||
for right in texts {
|
||||
let merged = reconcile(parent, &left.into(), &right.into(), &tokenizer).apply();
|
||||
for input in [left, right] {
|
||||
assert_eq!(
|
||||
reconcile(parent, &merged, &input.into(), &tokenizer)
|
||||
.apply()
|
||||
.text(),
|
||||
merged.text()
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ambiguous_ancestral_text_prefers_a_stable_input() {
|
||||
let parent = "a b ";
|
||||
let left = "a b a b ".into();
|
||||
let right = "c d ".into();
|
||||
let tokenizer = &*BuiltinTokenizer::Word;
|
||||
let result = reconcile(parent, &left, &right, tokenizer);
|
||||
let merged = result.apply();
|
||||
assert_eq!(merged.text(), "a b a b ");
|
||||
for input in [&left, &right] {
|
||||
assert_eq!(
|
||||
reconcile(parent, &merged, input, tokenizer).apply().text(),
|
||||
merged.text()
|
||||
);
|
||||
}
|
||||
assert!(
|
||||
result
|
||||
.apply_with_history()
|
||||
.iter()
|
||||
.any(|span| span.history() == reconcile_text::History::AddedFromLeft)
|
||||
);
|
||||
assert!(
|
||||
!result
|
||||
.apply_with_history()
|
||||
.iter()
|
||||
.any(|span| span.history() == reconcile_text::History::AddedFromRight)
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -65,7 +65,7 @@ fn test_document_one_way_with_serialisation() {
|
|||
|
||||
doc.assert_eq_without_cursors(
|
||||
&restored_left_operations
|
||||
.merge(restored_right_operations)
|
||||
.merge_with_tokenizer(&restored_right_operations, &*BuiltinTokenizer::Word)
|
||||
.apply()
|
||||
.text(),
|
||||
);
|
||||
|
|
|
|||
Loading…
Reference in a new issue