All articles

Similarity earns a look. Then the work begins.

A high similarity score says a gene's phenotypeThe observable clinical findings of a patient, recorded here as HPO terms with their status. resembles the patient's. It says nothing about the variant, and a number shaped like a verdict is believed.

The gene is first on the list with a similarity score of 0.91. The scientist writes strong candidate in the case note and begins the report. The variant is a missense change inherited from an unaffected father, in a gene whose disease mechanism is loss of function, at a population frequency that would make it one of the commoner causes of a very rare disorder if it were causal.

The gene fitted the phenotype. The variant fitted nothing. The number did the thinking, and the report was halfway written before anyone asked what the father's genotype meant.

What the field knows

Phenotype similarity measures a resemblance between two sets of terms (Köhler et al., 2009). When a patient's terms are compared with a gene's annotated disease features, the score describes how well the gene's known phenotype matches this patient's described phenotype. It is a statement about the gene and the description. It carries no information about the variant found in the gene, whether it is rare, whether it is predicted to damage the protein, whether it segregates, or whether the mechanism it implies is the one the disease requires. Two patients with the same phenotype and the same gene, one with a truncating variant and one with a common missense change, receive the same similarity score.

The tools that rank genes for a patient combine the phenotype score with variant-level scores for this reason (Smedley et al., 2015), and their authors have been consistent that the output is a ranking to review and the review is the diagnosis (Jacobsen et al., 2022). The ACMG framework (Richards et al., 2015) places phenotype match as one supporting criterion among many, and the variant criteria stand on their own evidence.

People read numbers as verdicts. The automation bias literature (Goddard et al., 2012) describes the pattern: when a system presents a ranked answer, users accept it more often than the evidence warrants and search less for disconfirming information. A score shown to two decimal places, at the top of a sorted list, is designed, whether anyone intended it, to be accepted. The more confident the display, the less the reader checks, and a sorted list with scores is a confident display, in a field where the reader is tired, the queue is long and the top answer is usually right.

Where practice breaks

The score is read as a probability. A similarity of 0.91 is understood as a nine-in-ten chance that this is the answer, when it is a relative measure whose scale depends on the ontology, the annotations and the other candidates in this list. The same gene with the same variant would score differently on a different day's annotations, or in a case where one more term had been coded.

The rank anchors the review. The top candidate gets an hour, the tenth gets a glance, and the answer in the twelfth position, a gene with a sparse disease annotation and a de novo truncating variant, gets nothing. Ranking by phenotype alone penalises exactly the genes whose phenotypes are least described, which are the genes most likely to be new. The list is sorted by how much is known, and the reader takes it as sorted by how likely.

The variant questions are asked after the gene has been chosen, and asked to confirm. Frequency, mechanism and segregation are checked for the top candidate in a frame of mind that wants it to pass. In the case above, three questions would each have removed the gene, and none was asked until the report was drafted, by which point the answer had a name and the questions had become objections to it.

Scores are compared across cases. A 0.91 in this case is treated as better evidence than a 0.74 in another, when the two are on scales set by two different term sets and two different candidate lists. The comparison has no meaning, and it is made in multidisciplinary meetings every week.

How Tulparus handles it

Phenotype similarity is one channel among several that propose candidates, alongside the variant's own evidence, the inheritance pattern in the family, gene-disease validity and similar cases. A candidate that one channel favours and another contradicts shows the disagreement, and the disagreement is the point at which a scientist looks. A candidate that only one channel proposes is marked as resting on one channel.

No score is shown as a chance of diagnosis. A candidate is shown with its rank in this case and the reasons for it: which of the patient's terms matched which of the gene's features, which did not match, and which disease features were never assessed in this patient. Beside it, as separate items with their own evidence, stand the variant questions: population frequency against the disease's prevalence, the mechanism the disease needs against what this variant does, and what the family shows. A gene at the top of the phenotype channel with a variant that fails two of these is displayed as exactly that.

The ranking changes nothing in the investigation until a person writes a hypothesis. The hypothesis names the gene, the variant, the mechanism and the inheritance, and the workbench lists what evidence would support or refute each part. The case above would have shown, before any note was written, that the father carries the variant, that the mechanism is loss of function and this is a missense change with no functional evidence, and that the frequency is too high. The score would still have been 0.91. It would have been one line among several, and the others would have been red, and the report would have been about a different gene.

Where the product shows how a ranking method has performed, it shows benchmark performance on a named evaluation set, with the release and the sample, labelled as a property of the method on that set and never as a statement about this patient.

What you can do tomorrow

The habit to build is asking the variant questions before the gene has been chosen, so that the answer has to earn its place twice. Each of the steps below can be done on the current tools, with a template and a rule.

  • Write the three variant questions, frequency, mechanism and segregation, on the review template above the candidate list, and answer them for the top five candidates before reading any phenotype score.
  • Remove the decimal places. A rank, or a band, carries the same information as 0.91 without inviting it to be read as a probability.
  • Review the bottom of the list once. Ask what a gene would have to lack in annotation to be ranked there and whether the patient's gene could be one.
  • Never compare a score across two patients in a meeting. If the comparison is tempting, the display is wrong.

Sources

  1. Köhler S, Schulz MH, Krawitz P, et al. Clinical diagnostics in human genetics with semantic similarity searches in ontologies. American Journal of Human Genetics, 2009.
  2. Smedley D, Jacobsen JOB, Jäger M, et al. Next-generation diagnostics and disease-gene discovery with the Exomiser. Nature Protocols, 2015.
  3. Jacobsen JOB, Kelly C, Cipriani V, et al. Phenotype-driven approaches to enhance variant prioritization and diagnosis of rare disease. Human Mutation, 2022.
  4. Richards S, Aziz N, Bale S, et al. Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genetics in Medicine, 2015.
  5. Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, 2012.

A score with its reasons.See a ranking that cannot be mistaken for an answer.

A demo takes 20 minutes, screen shared, on synthetic cases. Bring the question from one of your own.