A sequence of prompts across three weeks, in substance: evaluate whether Fugu-style orchestration would improve Narracode; assess whether the AI-tells review pass is optimally architected; ask how the idiomatic cadence of Narracode prose could be nuanced while encouraging psychologically astute, non-sensationalist work. Then: the diffs between machine draft and human edit constitute a corpus — the corpus is not my voice, it is the voice I was attempting to find in each instance — could something rapid, elegant and efficient be built from it? Then: is it possible to retroactively assess the temperature of already-written text? And throughout: we are not interested in rules, we are interested in harnessing recursive self-improvement for literary processes.
Between mid-July and early August 2026, alongside the ongoing writing, the Narracode project measured itself. The question was practical: every story in the repository exists as a machine draft that a human then edited. Those edits are a record of what a writer does to model prose. If the record could be read, the harness might learn from it.
In short
The edits were scattered. Some sat in version snapshots, some only in git history — early stories were edited in place and committed, so the machine original survives only as a parent commit. One complete story lived on an unmerged branch and had never been published.
Recovering them took six substeps: survey, provenance ledger, sentence segmentation, alignment, emission, validation. Provenance could not be automated. Directory names carry four different conventions and roughly half say nothing about who made the change, so 41 snapshots were resolved by hand. Guessing was refused throughout: a mislabelled pair inverts a measurement and nothing downstream would notice.
| source | pairs |
|---|---|
| version snapshots | 405 |
| published HTML vs final draft | 90 |
| git history | 739 |
| total, deduplicated | 1,234 |
Nine stories. The corpus validates against ground truth independent of the extractor, and rebuilds byte-identically.
Working artifact, preserved as a record: the provenance diff review (5 Aug 2026) — all 67 snapshot transitions across 12 stories, rendered chronologically, each asking whether the change came from the author or the model. This is the page the 41 hand-resolved provenance calls were made against. Alongside it, the Crepuscular diff lineage (6 Aug 2026), kept unedited though its premise no longer holds — it describes the story as unmerged two days before it was published as Upboarding Hour.
Six measures were proposed in advance. Three survived contact with the data.
| measure | direction | t | consistent |
|---|---|---|---|
| mean sentence length | shorter, −2.0 words | −4.36 | 95% |
| em-dash count | fewer, 158 → 114 | −5.72 | 97% |
| naming clauses | almost gone, 12 → 1 | −3.07 | 100% |
| figuration | less, 33 → 22 | −2.86 | 87% |
| sentence-length variation | null — 42% consistency | ||
| three-item lists | null — 73 → 74 | ||
Every surviving measure subtracts. Nothing the instruments can see is additive. This restates, in numbers, a finding the project reached by observation in May 2026: the active discipline of the system is what it forbids, not what it organizes. Two methods, one conclusion.
Two measures had looked strong and collapsed under control. Every dimension was originally specified as a rate per thousand words — but edited text is about 20% shorter, which inflates every rate by roughly a quarter for free. Recounted as raw counts, the apparent rise in three-item lists vanished and the em-dash effect tripled. The 90th-percentile jump between adjacent sentence lengths looked like the best result in the corpus until it was normalised by mean sentence length, at which point it disappeared. Shortening sentences shrinks the gaps between them arithmetically.
The plan had argued that sentence-length variation carries prose rhythm, on the reasoning that human writing is burstier than machine writing. It does not, here. Variation is flat. Length itself is what moves.
Of 689 edits with text on both sides, the surviving measures say something about 9%. The other 91% is rewriting they cannot see:
No rule states these. A full stop becomes a question mark; a common word becomes a strange one. This is the substance of literary revision and it was, on this evidence, largely untouched by any measurement the project had built.
A question was raised: could the sampling temperature of already-written text be recovered? Literally, no — temperature is a generation-time parameter, not invertible, and human edits are never sampled. But its effect, probability mass moved toward unlikely words, is measurable afterwards through word rarity.
It is the largest effect in the corpus. Words removed average Zipf 5.88; words added, 4.91 — roughly ten times less frequent. 84.5% of edits move rarer. The share of added words absent from any frequency list rises twenty-fold, because some are coinages:
This was reported as a human signature. That was wrong, and the check that broke it was available in the project's own bibliography. Voice Under Revision is the inverse experiment — three frontier models revising 300 human narratives, where this corpus is one human revising machine fiction. If human revision were distinctive, the two should diverge. Four of five comparable markers move the same way:
| marker | this corpus | LLM revision | |
|---|---|---|---|
| mean word length | +0.35 | +8.5% | same |
| type-token ratio | +2.95 | +53% | same |
| comma rate | +11.2 | +67% | same |
| first-person density | −0.46 | −9% | same |
| em-dash | −5.72 t | +326% | opposite |
Rarity, lexical diversity and word length are one family, and both the writer and the models move that family in the same direction. Rarity measures that a text was revised, not who revised it. The measurements stand; the interpretation does not.
The em-dash alone separates them, with large effects and opposite signs on both sides. It is a poor detector of machine authorship, which is how it is usually discussed. It is a good detector of machine revision.
One caveat the comparison deserves: the studies share no baseline. One revises casual speech, the other machine literary fiction. Both raise word length, but the mechanisms visibly differ — their commas arrive with added parentheticals, these arrive alongside sentences getting shorter. Same marker, opposite construction. And this corpus is a single writer with a deliberate poetics, not a population.
Human and machine revision are, on aggregate stylometry, largely indistinguishable — while producing prose no reader would confuse. That is the strongest available demonstration that aggregate stylometry does not reach the thing.
It does not close the project. The measurements were an attempt to build a theory of the voice, so a model could be told where to aim. That failed and should stay failed: a 9% instrument used as a target would optimise the shell and leave the substance alone.
The corpus did not fail. Those are separable objects, and only the first was tested. A
system does not need to understand an edit to improve from it — handing a model eight real
before → after pairs transmits their → craw without anyone naming why
it is better. The exemplars carry the 91% precisely because nobody had to theorise it.
So the next step is retrieval, not measurement: the nearest past edits injected before drafting. The test is fixed in advance and folds into work already planned — one story, scene arms assigned by coin flip, each scene seen once, the editing burden measured by the same pipeline that built the corpus. If the two arms are indistinguishable, this line of work has reached its limit, and the corpus stands as a documented negative result about the measurability of literary revision.
One risk names itself. Edits are the signal, so a system that reduces editing thins its own training data. Not fatally — as the model closes the distance, the edits become finer rather than fewer — but the index will need to weight recent work more heavily than old.
Three things are worth keeping. A validated corpus of 1,234 human edits to machine prose, with full provenance, which appears to be unusual. The em-dash as the one measure that distinguishes who revised a text. And a negative result: that literary revision is not captured by the stylometric instruments built to capture it, arrived at by building them carefully enough to be sure.
Two claims were made and withdrawn in the course of three weeks — that sentence-length variation carries the signal, and that word rarity is a human signature. Both were withdrawn by controls rather than by argument. That sequence is kept in the repository deliberately.
David Jhave Johnston is a digital poet working in emergent domains. Author of ReRites (Anteism, 2019) and Aesthetic Animism (MIT Press, 2016). He is currently an AI-narrative researcher at the UiB Centre for Digital Narrative (2023–27) with the Extending Digital Narrative project.
This work was partially supported by the Research Council of Norway through its Centres of Excellence scheme, project number 332643 (Center for Digital Narrative), and its SAMKUL project scheme, project number 335129 (Extending Digital Narrative).
