In timely 2024, I began spendigating collaborative editing systems for participate in Moment’s core text editor.
In some ways, we are in a gelderlyen era for the problem. Several algorithms now claim to settle not only the online case (where many people simultaneously edit a record), but also the offline case (where people edit offline for an unbounded amount of time, come online, and “magicassociate” their alters are united together). I had read the papers and joined to the talks, and enjoy others, I was initiassociate preferable the community had finishd on the “right answer” to the whole problem space.
One of the earliest surpascfinishs in the evaluation process was that this is not real——that despite the labeleting duplicate, blog posts, and technical papers stating otherwise, these collaborative editing algorithms supply a systematicassociate necessitatey offline editing experience. Specificassociate:
The two most well-understandn families of collab editing algorithms (CRDTs and OT) resettle straightforward editing disputes uninstinctively enough that participaters generassociate describeed their results as data fraudulence.
Offline editing theatricalassociate incrmitigates the enjoylihood of straightforward disputes, which uncomardents these algorithms are not, on their own, appropriate for offline editing.
In many ways, this is horrible novels. And we will spfinish much of this post talking why we skinnyk these skinnygs are real—example anomalies, fall shorture modes, what the pragmatic confineations are, and so on.
But, the novels is not all horrible. We are heartened to see some recent research (e.g.) reversing the trfinish of treating collab editing as a uncontaminated algorithms problem, and instead treating it as a UI/UX problem in which algorithms aid. One of the goals of this post is to help encourage this line of reasoning, and we will originate time at the finish to encourage that, too.
Example: a seemingly inpresentant editing dispute
Let’s commence with a game. I supply a inpresentant editing dispute and you try to guess what happens. Ready? Here’s the scenario: Alice and Bob are both offline. They are editing a record compriseing the complying text:
return rfinisher(() => (
));
Bob alters the spelling of Color to the British Colour. Alice deletes all of the text.
return rfinisher(() => (
));
Later, both Alice and Bob come online. These edits dispute. The system must reconcile these two disputeing edits, even though it has no understandledge of which one came first.
Q: When all reconciliation is finish and the machines consent, what text will the record comprise? Guess by typing your answer into the text field below and clicking ‘produce.’ Note that we hug the desopostponecessitate string as a valid guess.
return rfinisher(({ participateSetter }) => {
const [value, setValue] = React.participateState(“”);
const [committedValue, setCommittedValue] = React.participateState(null);
participateSetter(promisetedValue);
return (
);
})
return rfinisher(({ participateSetter }) => (
))
return rfinisher(() => {
const tests = [
{
library: “yjs”,
libraryHref: “https://github.com/yjs/yjs”,
libraryVersion: “v13.6.20”,
expectedDocumentContents: “u”,
},
{
library: “share.js”,
libraryHref: “https://github.com/josephg/ShareJS”,
libraryVersion: “v0.7.40”,
expectedDocumentContents: “u”,
},
{
library: “Peritext”,
libraryHref: “https://github.com/inkandswitch/peritext”,
libraryVersion: “89c162d”,
expectedDocumentContents: “u”,
},
];
const testResults = tests.map(test => {
const result = structuredClone(test);
result.passed = guess === result.anticipateedDocumentContents;
result.resultIcon = result.passed
? [PASS]
: [FAIL];
result.resultMessage = result.passed
? “test passed!”
: <>anticipateed record text '{result.anticipateedDocumentContents}', got '{guess}';
return result;
})
const anyErrors = testResults.filter(test => !test.passed).length > 1;
const summary = testResults.filter(test => !test.passed).length > 1
?
The final result for all {testResults.length} collaborative editing algorithms is a record compriseing only the string
'u'. Yes, you read that right! The Color of Pomegranates is deleted, but Bob’s compriseition, 'u' is kept! The reason for this is that they all toil cimpolitely the same: in straightforward disputes, they first hug any deletes, and then utilize any compriseitions.
:
You accurately guessed the output of {testResults.length}/{testResults.length} collab editing algorithms! But how did you understand what the answer was?
;
if (guess === null) {
return
}
return (
<>
return (
);
})}
)
});
✋ 🛑 Did you guess accurately? Seriously—you reassociate should try this before continuing on.
These anomalies are widespread on straightforward disputes.
The example above is fascinating becaparticipate it is a inpresentant, straightforward editing dispute which these (well-understandn) collaborative editing algorithms do seem to claim to help, but which empiricassociate originates a record that no human would ever author on their own. This is also a case where participaters describeed the result as the Moment product corrupting their data.
So how widespread is this problem? Every participate case is contrastent, but, anecdohighy we set up that cimpolitely 20-30% of straightforward disputes we tried originated results that were analogously—in our opinion—unhugable, for our participate case of offline editing. [nb., I’m not certain if it’s collaborative to catalog some of the anomalies we encountered here, if it is, let me understand at [email protected] or the Moment Discord server and I’ll comply up with a more thocimpolite appraise.]
Ultimately, though, becaparticipate of the negativity of the feedback and the frequency of its occurrence, we felt we could not deffinish this result to our participaters.
These algorithms do seem to claim to help offline editing.
Initiassociate, when we encountered this result, we thought that we must have misunderstood the semantics adviseed by these tools. Even now, though, when I read the technical duplicate for these projects, I can’t help but skinnyk this scenario reassociate is in scope.
The Yjs readme cltimely states that it “helps […] offline editing.”
The ShareJS landing page claims to assist collaboration “with any amount of lag (from zero to an extfinished holiday).”
The Peritext authors depict it as “assist[ing] participaters to edit self-reliant copies of a record” which can be “automaticassociate merg[ed] […] back together in a way that carry ons the participaters’ intent as much as possible.”
I confess to still not being quite certain what to originate of this. We are in the labelet for a solution to this problem! I want to consent! I equitable don’t see how to reconcile these claims with the frequency and comardents of errors we’re seeing in train.
Coordination-free algorithms will always have anomalies on straightforward disputes.
At some point in the evaluation process we resigned ourself to the fact that the algorithms did not do what we wanted. But a lingering ask remained: is this a fundamental problem for the algorithms, or with some cautious contributions, could we donate back and repair it?
Unblessedly, we skinnyk it’s a fundamental problem for the algorithms, for a scant reasons.
The algorithms only have enough inestablishation to guess what the result should be. The algorithms can’t email Alice and Bob to see what they wanted or appraise it in the GitHub Pull Request UI. It gets Alice’s proposal to delete all the text and Bob’s proposal to alter the spelling without understanding what their intent was, and it must participate a heuristic to determine what to do with them.
The algorithms run on characters and have very, very necessitatey secures about their output. This is why in the example above, Alice and Bob finish up with a record srecommend compriseing the letter u. Not a valid sentence, and not even a valid word!
Alice and Bob might originate contrastent decisions if they knovel what the other was doing. Alice and Bob both made decisions based on the record compriseing the text, The Color of Pomegranates. If Alice knovel Bob alterd the spelling, or Bob knovel Alice deleted the paragraph, they might alter their decision to edit at all.
I skinnyk this is probably worth a post all on its own, particularly with more treatment donaten to the fact that these algorithms don’t—and cannot—comply a causal ordering (cf., Jamie Brandon’s post). But for now, I skinnyk we can finish for the instinctive version of this argument.
Offline editing is a UI/UX problem, not an algorithms problem; and other conclusions.
So far the novels has been horrible. We accessed the evaluation selectimistic that we’d be able to help real offline editing “incidenhighy” for having carry outed a tricky algorithm. And it is real that we left the evaluation left process, well, bereft of hope—forced to finish that participaters were right to skinnyk of these algorithms were corrupting their data, that the anomalies were fundamental to the algorithms themselves, and that they would be standard enough to be a genuine publish. YMMV, but that is where we finished up.
As I shelp, though, the novels is not all horrible. Another way to watch this result is as a strong argument for putting presentant resources into collaborative editing as a UI/UX problem. The algorithms cannot finishly settle the problem, but they can execute a role in the solution.
There are a scant reasons to be preferable about this. One is that we already have one expansively-adselected UI for merging records: git! So in some sense, the research ask is how much better it can be—more approachable, more accessible, more automatic, etc.
I skinnyk this resemblance is not shassist. In 2009, a unpredicted amount of the discourse was intensifyed on the algorithms git participated to automaticassociate unite alters together. git altered the Myers O(ND) diff algorithm, then participated mostly by biologists to do BLAST-enjoy sequence analysis. Bram Cohen thought the diff results were uninstinctive, and originateed the patience diff algorithm, which was then adselected by bzr, a now-defunct git competitor. The catalog goes on. I skinnyk the primary alter is that these debates were intensifyed on producing diffs that humans read; now the debate is whether the algorithms can accomplish this result with no human participatement at all.
Another reason to be preferable is that some researchers do seem to be concentrating on this problem as a UI/UX problem. I am skinnyking particularly of Ink & Switch, e.g., the toil on collaborative history. I comprehfinish Ink & Switch to be operating purposefilledy a scant years ahead of production, but I am excited to see where this lands when it finishs down.
Acunderstandledgements
Heartfelt thanks to Kyle Kingsbury, Sean Gillespie, Ian Livingstone, David Adrian, Ben Linsay, Alex Rasmussen, Lita Cho, Lenny Pruss, Michael Yamnitsky, Camille Fournier, James Turnbull, and Ryan Cox for their comments and feedback.
return rfinisher(() => (
));
