A clinician asks a laboratory why a variant reported as likely pathogenic two years ago is listed in ClinVar today as of uncertain significance. The scientist who wrote the report has left. The report says 'ClinVar: likely pathogenic'. It does not say which submitter, which release, or whether other submitters disagreed at the time. A sibling has already had carrier testing on the strength of it.
The laboratory cannot tell whether its own interpretation was wrong, whether the evidence changed, or whether a single submitter's downgrade moved an aggregate that was always thin. The conversation with the family waits on an archaeology.
What the field knows
ClinVar (Landrum et al., 2014; 2018) is an archive of submitted interpretations. Each record has a submitter, a date, a review status and, in many cases, the evidence behind it. The archive aggregates records into a per-variant summary and is released in full on a regular schedule, each release carrying a date. It grows and changes: submitters add records, update them and occasionally withdraw them, and the aggregate for a variant moves when any one of them does.
Disagreement among submitters is a documented property of the archive. Yang and colleagues (2017) examined variants with more than one submitter and found a measurable share with conflicting classifications, concentrated among older submissions and those made under different guidelines. Harrison and colleagues (2017) describe laboratories in a ClinGen working group resolving many of their differences once they shared internal evidence, which shows that the conflicts are often about what each laboratory had seen rather than about the variant itself.
ClinGen (Rehm et al., 2015) added expert panels whose classifications carry a higher review status, and ClinVar's star rating marks them, so that a reader can weigh a three-star expert consensus differently from a single zero-star submission. The information needed to read a classification carefully is in the archive. It has to be carried out of it.
A classification is therefore a statement about a variant at a date, from a submitter, at a review status. Quoted without those, it is a different statement from the one the archive makes, and it cannot be compared with the archive later. The same holds for every curated source a laboratory uses: population frequencies change between versions, gene-disease validity is revised, ontology terms are merged and retired. ClinVar is the clearest case because its aggregate is the one most often quoted in a report.
Where practice breaks
Annotation pipelines flatten it. A typical pipeline annotates each variant with a single ClinVar field, usually the aggregate clinical significance, taken from whichever release was downloaded when the pipeline was last updated. The release date is in a file name on a server; the submitter list and the star rating are dropped. The report inherits the one word.
The release in use is often old, and nobody knows how old. Reference data are updated when someone remembers, and the update is a change to a production pipeline, so it is done carefully and rarely. Two laboratories reporting the same variant in the same month may be reading releases a year apart, and quote different aggregates with equal confidence.
Conflicts are hidden by the aggregate. 'Conflicting interpretations of pathogenicity' is a legitimate ClinVar value, and a pipeline that maps it to a colour or drops it loses the one thing a scientist most needs to know: that someone disagreed, and who. A scientist who sees only 'likely pathogenic' from a single submitter in 2016 reads it as the field's view.
The report is where the loss becomes permanent. The scientist may well have read the submitter list on the website at the time and weighed it properly. The report carries the word, the laboratory's internal record carries the word, and the website carries only today. The reasoning that was done is not recoverable by anyone, including the person who did it.
Reanalysis then cannot be done well. To know whether a case should be reopened, a laboratory has to know what it looked at the first time. If the first look is recorded as a word, there is nothing to compare. If it is recorded as a release, a submitter and a status, the laboratory can ask what changed between then and now, and act on the cases where the answer matters.
How Tulparus handles it
The workbench never stores a classification without its provenance. Every piece of knowledge used in a case is recorded with its source, its release and its record identifier, and ClinVar is handled as a sequence of releases, each loaded whole and kept. A classification on a case reads as: this variant, in this release dated so, had these submissions at these review statuses, aggregating to this. The submitter-level records are kept, so a conflict is visible as a conflict.
Conflicting evidence is never overwritten. If a later release changes a submitter's classification, the earlier record stays in the case history with its date and the later one is added beside it. The case shows that the aggregate moved, when, and because of whom. Nothing a scientist relied on disappears from the record because the archive moved on.
The scientist reviewing a variant sees the submissions laid out by review status and date, with the laboratory's own earlier classification of the same variant, if there is one, beside them. A change in the archive and a change in the laboratory's view are two different events, and the case keeps them apart.
Each investigation records the knowledge release it was performed against, with the case version and the model version. When a new ClinVar release is loaded, the workbench computes the difference against the release each case used and lists the cases in which a variant that was considered has changed classification or gained a conflict. That list is the reanalysis queue, with the reason attached to each entry.
The report is rendered from this record, so the word in the report comes with its date and review status. A clinician reading it in three years can see what was known, and the laboratory can answer the question in the opening case in a minute rather than a week.
What you can do tomorrow
Most of this is bookkeeping, and the bookkeeping can start before any tooling changes.
- Find the ClinVar release your pipeline is using today and write the date into your report template. If finding it takes more than a few minutes, that is the first problem.
- Carry the review status and the number of submitters into the annotation, beside the aggregate.
- Treat 'conflicting interpretations' as a flag for review rather than a value to map away.
- For every reported variant, keep the release that was used in the case record, so that the next release can be compared with it.
Sources
- Landrum MJ, Lee JM, Benson M, et al. ClinVar: improving access to variant interpretations and supporting evidence. Nucleic Acids Research, 2018.
- Landrum MJ, Lee JM, Riley GR, et al. ClinVar: public archive of relationships among sequence variation and human phenotype. Nucleic Acids Research, 2014.
- Yang S, Lincoln SE, Kobayashi Y, Nykamp K, Nussbaum RL, Topper S. Sources of discordance among germ-line variant classifications in ClinVar. Genetics in Medicine, 2017.
- Harrison SM, Dolinsky JS, Knight Johnson AE, et al. Clinical laboratories collaborate to resolve differences in variant interpretations submitted to ClinVar. Genetics in Medicine, 2017.
- Rehm HL, Berg JS, Brooks LD, et al. ClinGen: the Clinical Genome Resource. New England Journal of Medicine, 2015.