Forensic error audits of published guidelines and research
Numbers that do not reconcile, statistics that cannot be true of the data shown, definitions that cannot exist, and citations that do not say what the text claims. Every finding is checked against the source, and the authors or issuing body are told before anything is published here.
Published audits
Critical: followed or acted on as published, the document could plausibly harm a patient. Major: no direct safety consequence, but the error affects a recommendation, a conclusion, a headline number, a definition, or a load-bearing citation.
Method
Every audit follows the same written procedure. Detection is AI-assisted; nothing is published until a physician has re-derived it from the source.
What is audited
Published documents only: clinical guidelines, committee opinions, practice bulletins and society statements, and original research, systematic reviews, meta-analyses and research letters. Unpublished or under-review manuscripts are never audited here.
How severity is assigned
Each document is first classed as guidance, which is followed as written, or research, which is believed and acted on. Severity follows the consequence of the error, not its size.
| Critical | Followed or acted on as published, the document could plausibly harm a patient: a wrong dose, unit or threshold, a reversed direction of effect, a safety conclusion contradicted by the document's own harms data, or a universal recommendation that omits a standard contraindication. |
|---|---|
| Major | No direct safety consequence, but the error affects a recommendation, a conclusion, a headline number, a definition the document relies on, or a load-bearing citation. |
| Minor | Correctable without affecting any recommendation, conclusion or clinical decision. Minor findings are counted in each audit. |
What the audit checks
- Identity and correction status: the version audited, and any errata, retractions or superseding editions.
- Arithmetic and reconciliation: every sum, percentage, rate, denominator, conversion and unit, recomputed from the document's own numbers.
- Statistical compatibility: whether p values, confidence intervals, test statistics and effect estimates can all be true of the data shown.
- Internal consistency: abstract against body, text against tables and figures, recommendations against the caveats stated elsewhere in the same document.
- Definitions: every classification, criterion, threshold and named entity against its authoritative definition.
- Method fidelity: what was reported against the protocol, registry record or stated method.
- Citation fidelity: whether each load-bearing source says what the document says it says, checked against the source itself.
- Evidence to claim: whether each recommendation or conclusion is supported by the evidence the document offers.
Each finding carries one error type: internal contradiction; arithmetic, unit or reconciliation failure; statistical incompatibility; definitional or taxonomic error; claim exceeds evidence; deviation from protocol or stated method; undisclosed missing data or misclassification; bias check misreported; citation misrepresents source; or editorial integrity. Findings typed as claim exceeds evidence involve weighing evidence, and are labeled so that readers can tell a judgment from a verifiable error.
How findings are verified
A finding is published only when it is provable from the document alone (verified internal) or confirmed against a retrieved primary source that is cited and quoted (verified external). Anything that cannot be settled from the published record is dropped. Values read from figures are treated as approximate. Every critical and major finding then goes through an adversarial pass, in which the audit argues the authors' side against its own findings; findings that do not survive are withdrawn, and those that survive in narrower form are published in that form. Finally, the registry editor re-derives each finding from the source document.
Notification and response
The authors or issuing body are notified before an audit is published and given 30 days to respond. A response is shown beside the findings it addresses. When a response shows that a finding is wrong, the finding is withdrawn and the reason stated.
| In notice window | Notified; not yet public |
|---|---|
| No response | The window closed without a reply |
| Acknowledged | The authors or issuer agree; no correction yet |
| Corrected | An erratum or revised version has been issued and is linked |
| Disputed | The authors or issuer disagree; their response is shown |
| Withdrawn | We withdrew the finding; the reason is stated |
Corrections to this site
Audit pages are never deleted. A withdrawn finding stays on its page, marked withdrawn, with the reason and the date, and every change after publication is logged on the page.
Scope and disclosure
Audits critique documents, never their authors. The registry editor is an editor of the American Journal of Obstetrics and Gynecology; errors found in AJOG papers are raised through AJOG's own correction process first.