All articles

When two laboratories disagree about a variant.

The same variant is pathogenic in one submission and uncertain in another. The disagreement is information, and one label in its place throws it away.

A variant was classified pathogenic by one laboratory in 2016. The family were counselled on that basis in 2017 and used the result for prenatal testing. A second laboratory submitted the same variant as uncertain in 2019, having seen it in two unaffected adults in its own cohort. A third called it likely benign in 2022, after population data showed a frequency the original classification had not had access to.

The family have never heard about 2019 or 2022. The 2016 report is in the notes, and it is the one anyone reads. The variant has been downgraded twice in the public record and nobody in the family's care has looked.

What the field knows

Disagreement between laboratories about the same variant is common and well studied. Amendola and colleagues (2016) had laboratories in a consortium classify the same variants independently and found substantial discordance, much of it resolved on discussion once each laboratory saw the others' evidence. Yang and colleagues (2017) examined the sources of discordance in ClinVar and found that differences in the evidence available to each submitter, differences in the thresholds applied to the same evidence, and differences in the date of assessment each accounted for a share. Harrison and colleagues (2017) showed that laboratories can resolve a majority of their differences when they share the underlying evidence and reassess.

ClinVar records the disagreement rather than resolving it. Each submission carries its submitter, its date, its classification and, when provided, its evidence, and the aggregate record shows conflicting interpretations when submitters differ (Landrum et al., 2018). The review status marks whether an expert panel has reviewed the variant. ClinGen's expert panels exist to produce that reviewed classification for genes where the stakes are high (Rehm et al., 2015), and their classifications are explicitly dated and versioned because they change. A classification is an opinion on a date, and the field has slowly built the infrastructure to say so.

The reasons for discordance are informative. A laboratory that has seen the variant in unaffected individuals in its own cohort knows something the others do not. A laboratory that classified before a population database release was working without data that now exists. A conflict in which the dates and the evidence are visible is a conflict that can be understood, and often one that can be resolved. A conflict shown as a single word is neither.

Where practice breaks

The conflict is collapsed. A pipeline that needs one label per variant takes the most recent, or the most severe, or the majority, and the case record shows one word where the database shows three dated opinions. The scientist reading the record does not know that a disagreement exists, and so cannot ask why, and so cannot learn that one submitter had cohort data the others lacked.

The dates are ignored. A 2016 pathogenic and a 2022 likely benign are shown side by side as if contemporaneous, when the later classification had six more years of population data and the earlier one may simply be out of date. The earlier laboratory was not wrong on the day; it is wrong now, and the record does not say which. A classification is a statement made at a date on the evidence of that date, and read without its date it is read wrongly.

The laboratory's own classification is not re-examined when others disagree. The 2016 laboratory, if it noticed the 2019 submission, would have had reason to look again; most do not notice, because nobody is watching the variant after the report goes out. The report is the end of the laboratory's process and the beginning of the family's, and nothing connects the two afterwards.

The family is not told. A result used for reproductive decisions was contradicted twice in the public record and the record in the family's notes is the original. Who should tell them, and when, is a question most services have never had to answer, because the systems that would raise it do not exist. The question arrives, when it arrives, as a complaint, years later, from a family who found the public record themselves.

How Tulparus handles it

Every classification is kept with its submitter, its date, its review status, its evidence where provided and the ClinVar release in which it appeared. Nothing is collapsed. A variant with three dated opinions is shown with three dated opinions, in date order, and the evidence store does not overwrite any of them when a newer one arrives. The release that carried each opinion is recorded, so that a scientist can ask what was known in 2016 and get the 2016 answer.

A conflict is a question in the investigation. When submitters disagree, the workbench raises it as an open item on the case: what each submitter saw, what has changed in the population data between their dates, and which piece of evidence would resolve the disagreement. The question stays open until a person closes it, and the closing is recorded. The laboratory's own classification is recorded in the same way, with the criteria it applied and the date, so that it can be compared with the others on equal terms.

A change in any submission on a reported variant is an event. When a new submission arrives on a variant that was reported in a case, or an existing one is revised, the case is flagged with the specific change, and the service is asked whether the report needs a revision and whether the family should be contacted. The decision, either way, is recorded with the person and the date, so that a service can show what it knew, when, and what it chose to do.

For the family above, the 2019 submission would have reached the case within the knowledge release that carried it, the 2016 classification would have been set beside it with the evidence of each, and the service would have been asked, in 2019 and not never, what it wished to do.

What you can do tomorrow

The first step is to stop collapsing the conflict, and the second is to watch the variants you have reported. Neither needs software; both need a decision that somebody owns the variant after the report.

  • When a pipeline shows one ClinVar label, open the record and look at the submissions with their dates before writing anything. Teach trainees that the aggregate label is a summary, and a lossy one.
  • Record the date and the evidence of your own classification in the report, so that a reader in six years knows what you did and did not have.
  • Keep a list of reported variants and check them against each ClinVar release. The list is a spreadsheet; the check is an afternoon a quarter.
  • Decide now, in writing, who contacts a family when a reported variant is downgraded, and how quickly.

Sources

  1. Amendola LM, Jarvik GP, Leo MC, et al. Performance of ACMG-AMP variant-interpretation guidelines among nine laboratories in the Clinical Sequencing Exploratory Research consortium. American Journal of Human Genetics, 2016.
  2. Yang S, Lincoln SE, Kobayashi Y, Nykamp K, Nussbaum RL, Topper S. Sources of discordance among germ-line variant classifications in ClinVar. Genetics in Medicine, 2017.
  3. Harrison SM, Dolinsky JS, Knight Johnson AE, et al. Clinical laboratories collaborate to resolve differences in variant interpretations submitted to ClinVar. Genetics in Medicine, 2017.
  4. Landrum MJ, Lee JM, Benson M, et al. ClinVar: improving access to variant interpretations and supporting evidence. Nucleic Acids Research, 2018.
  5. Rehm HL, Berg JS, Brooks LD, et al. ClinGen, the Clinical Genome Resource. New England Journal of Medicine, 2015.

Three opinions, three dates.See a conflict kept rather than averaged.

A demo takes 20 minutes, screen shared, on synthetic cases. Bring the question from one of your own.