All articles

Immutable evidence, versioned interpretation.

The facts in a case do not change. What they mean does. A system that edits records in place loses the first and cannot track the second.

A variant scientist updates a case after a new population release. The frequency field for the lead variant is overwritten with the new value, higher than before. The classification field is changed from likely pathogenic to uncertain significance. The report is regenerated. A week later the clinical team asks what the frequency was when the original report was issued, and whether it was below the threshold the laboratory used at the time.

The old value is gone. The scientist's best answer is a guess from memory and a search through an old browser cache.

What the field knows

Two different kinds of thing live in a case record, and they change in different ways. Evidence is what was observed or retrieved: a read depth, a genotype call, a frequency from a named release, a submission in a classification archive on a date, a sentence in a clinic letter. Once recorded with its provenance, a piece of evidence is a fact about the past and does not change. Interpretation is what the evidence is taken to mean: a classification, a hypothesis, a ranking, a report. Interpretation changes whenever the evidence grows or the rules move.

The interpretation frameworks are explicit that classifications are revisable and that the evidence behind them should be kept. The ACMG and AMP guidelines (Richards et al., 2015) build a classification from itemised evidence criteria, each of which can be traced to a source; the reanalysis guidance (Deignan et al., 2019) assumes that the earlier evidence and earlier interpretation are both available for comparison; reclassification studies (Mersch et al., 2018) can only be done at all because the laboratories concerned kept the earlier classifications beside the later ones.

The data management literature calls the pattern an append-only or event-sourced record. New facts are added; nothing is overwritten; the current view is derived from the full history. The FAIR principles (Wilkinson et al., 2016) ask for provenance on every item, which is only meaningful if the item survives. Clinical laboratory standards (Rehm et al., 2013) require retention of the data underlying a report, which in practice is honoured for raw files and often not for the intermediate evidence a scientist actually used.

The clinical reason to keep both is accountability across time. A classification is defended by the evidence available when it was made. If the evidence has been overwritten with today's values, the defence is impossible, and a reasonable decision looks like an error. The reverse also holds: a decision that was careless at the time can be dressed up afterwards if nobody can show what the evidence said then.

Where practice breaks

Most case systems are built as forms. A field holds the current value; updating the case means changing the field; the previous value is gone unless an audit log happened to capture it, and audit logs capture who changed a field, not usually what the field said or why. The design treats a frequency like an address, something that has a current correct value, when it is a measurement from a named source at a named time.

Evidence and interpretation are stored in the same fields. The classification cell sits next to the frequency cell, and both are edited in place, so the system cannot tell that one is a fact that should never change and the other is a judgement that should be versioned. When a scientist updates the judgement, the habit of the form invites updating the facts beside it.

Conflicting evidence is resolved by replacement. A new release disagrees with an old one; the new value wins and the old disappears. Two submitters disagree; the aggregate is stored and the disagreement is lost. The case ends up holding one tidy value where the truth was two values with dates and sources, and the scientist who must later explain the decision cannot show what was in conflict.

Reports are then regenerated from the overwritten state. The new report is right for today. The old report, if kept, cannot be reconciled with any stored case state, because the state it was rendered from no longer exists. The laboratory holds a document it can no longer explain.

The loss is felt most at reanalysis. A second look is meant to find what has changed, and that is only possible against a fixed record of what was. A case that was updated in place for three years has no earlier state to compare against; every reanalysis of it starts from scratch, and the earlier reasoning, which took days, is repeated or ignored. The scientist doing the second look cannot tell which candidates were dismissed on evidence and which were never reached.

How Tulparus handles it

Evidence in the workbench is immutable. A frequency, a classification submission, a phenotype finding, a read-level observation, each is written once with its source, release, record identifier and time, and never edited. A new release produces new evidence rows beside the old ones. A clinician correcting a phenotype term supersedes the earlier finding with a new one and the earlier one stays, marked as superseded, with the reason.

Interpretation is versioned. Hypotheses, classifications, rankings, investigations and reports are each a sequence of versions, every version pointing at the exact evidence and knowledge releases it was derived from. Changing an interpretation creates a new version; the previous version is kept whole. The case, viewed at any past moment, is reconstructable, because nothing it was built from has been altered.

Conflicts are first-class. When two sources or two releases disagree, both pieces of evidence sit on the case with their provenance, the conflict is shown as such, and any decision to prefer one is itself a recorded, versioned interpretation with an author and a reason. Nothing is overwritten to make the page tidier.

For the scientist in the opening case, the new release would have added a second frequency beside the first, each dated and sourced. The reclassification would have been a new version of the classification pointing at the new evidence, and the question from the clinical team would have been answered by opening the earlier version.

What you can do tomorrow

The principle can be applied to a spreadsheet as well as to a database.

  • Never overwrite an evidence value. Add a row with the new value, its source and its date, and leave the old row in place.
  • Keep interpretation in separate columns or tables from evidence, so that changing a classification does not invite changing the facts it rested on.
  • When two sources disagree, record both and write down which one you used and why. The disagreement is part of the evidence.
  • Before regenerating a report, snapshot the case state it will be rendered from, so that each issued report can be matched to the state that produced it.

Sources

  1. Richards S, Aziz N, Bale S, et al. Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genetics in Medicine, 2015.
  2. Deignan JL, Chung WK, Kearney HM, Monaghan KG, Rehder CW, Chao EC. Points to consider in the reevaluation and reanalysis of genomic test results: a statement of the American College of Medical Genetics and Genomics (ACMG). Genetics in Medicine, 2019.
  3. Mersch J, Brown N, Pirzadeh-Miller S, et al. Prevalence of variant reclassification following hereditary cancer genetic testing. JAMA, 2018.
  4. Wilkinson MD, Dumontier M, Aalbersberg IJ, et al. The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 2016.
  5. Rehm HL, Bale SJ, Bayrak-Toydemir P, et al. ACMG clinical laboratory standards for next-generation sequencing. Genetics in Medicine, 2013.

Facts kept, judgements versioned.See a case at any past moment.

A demo takes 20 minutes, screen shared, on synthetic cases. Bring the question from one of your own.