Validation
Tested on published cases.Reported with limits.
We measure each model on published cases it has not seen, and report the result with the evaluation set, the number of cases and the interval. A benchmark result describes a release. It is not the chance for your patient.
How we test
Held out, then frozen.
Cases used to tune a model are not used to test it. For new-gene tests, knowledge is frozen at a cut date, and only cases whose gene was not yet linked to disease on that date count.
Held-outevaluation cases are published solved cases the model has not seen, with phenotype and variants entered as reported
Frozenknowledge for new-gene tests, and only genes not yet linked to disease on the cut date count
Per releaseresults, published with every release. A model with a lower training loss never replaces the one in production
We show no headline numbers until the benchmark card for the current release is published. Every figure we report carries its evaluation set, number of cases, 95% interval, release and knowledge snapshot, and is benchmark performance, not the chance for this patient.
What we report
Per model, per case type.
Phenotype rankerRank of the causal gene from the phenotype alone, on published cases.
Genotype rankerRank with variants added, and what changes when parental genotypes are missing.
New-gene rankerRecovery of genes linked to disease after the cut date, within the candidate list only.
Note readerAgreement with terms entered by clinicians, and how often hedged findings are flagged correctly.
LimitsA result is withheld when a case type has fewer than 30 cases or 10 genes. Sparse phenotypes and non-coding findings are where we expect the weakest results.
Model cards
Ask for the details.
We share the current model card and benchmark card with qualified teams, including the knowledge release used for each run.