Open Review as Infrastructure
Signed, versioned critique attached to the work itself
This event is part of a demonstration of what Lera offers for running knowledge events. Its content is written with care and draws on real sources, but the conference has not taken place, and apart from the convener its participants are fictional.
Peer review in its usual form is an event, not an object. Letters pass between editor and reviewer, a file is attached, a decision is made, and then everything vanishes into inboxes and publishers’ closed systems. The reader of the published article never learns what objections were raised, how the author answered, or which parts of the text came out of that exchange. The most knowledge-bearing part of the scientific process, criticism, is the least recorded.
“Open review” is a well-known remedy, but it has no single definition. In 2017 Tony Ross-Hellauer examined more than a hundred definitions of it and showed that the term covers various combinations of seven traits: open identities of reviewers and authors, publication of review reports, open participation, direct interaction, publication of the manuscript before review, commenting after publication, and open platforms. This variety is not a weakness. It shows that open review is not one rule but a set of design decisions.
In 2023 the journal eLife made one of the boldest of these decisions. In 2022 Michael Eisen and colleagues had described a model in which, after review, the journal no longer accepts or rejects a paper; it publishes the reviewed version together with the review reports and a short assessment by the editors, and leaves the judgement of the work’s worth to readers and the scholarly community. Review moved from gatekeeping to public conversation. It was and remains controversial, but it proved that the thing can be done.
My argument is that before open review is an editorial policy, it is a matter of infrastructure. A critique has to be an object in its own right: it needs an identifier, a link to a specific version of the work, its writer’s name, the author’s reply kept beside it, and the possibility of being translated like any other text. The link that ties a critique to a version is vital: without it, a criticism of the first version may be wrongly attributed to the corrected one. And this infrastructure must be open source and governed by the scholarly community, not owned by publishers who profit from keeping things closed.
In the framework of language-agnostic knowledge this structure has a name: the main text of a work is a stable, versioned body, and reviews, replies, translations and references form a living orbit around it. Criticism stays in the orbit and has a history; the body changes only by human decision, in a new version. This is exactly what open review needs.
Questions for discussion
- Who should maintain a shared infrastructure for open review, and who should pay for it?
- Does publishing criticism make reviewers more cautious and more polite than they should be?
- How should a critique written in a language other than the work’s be kept beside the work?
References
Ross-Hellauer, Tony. 2017. “What Is Open Peer Review? A Systematic Review.” F1000Research 6: 588.
Eisen, Michael B., et al. 2022. “Peer Review without Gatekeeping.” eLife 11: e83889.