Internal and external accuracy are both true. Hiding the external gap is how models earn false trust.
Provotics classifies a tumor's site of origin at macro-F1 0.940 on a held-out test set. On the most adversarial test, a single tumor sequenced at a different centre on a different pipeline, raw accuracy drops to about 0.69, which is exactly why the model abstains on inputs it doesn't recognize rather than guess. Both numbers are real. The temptation is to print the first and quietly forget the second.
We do the opposite, and it is a deliberate design choice rather than modesty.
Held-out accuracy tells you how well the model does on data drawn from the same distribution it learned from. External accuracy, on tumors processed at different centres with different pipelines, tells you how well it generalizes. The gap between them is not a flaw to hide; it is the single most useful number for anyone deciding whether to trust a prediction on their own samples.
The failure mode we care about most is a confident wrong answer. So when a profile looks unlike anything in training, Provotics abstains and flags it as out-of-distribution instead of forcing a label. In testing, the cases that drove the external gap were largely caught this way, flagged, not mislabelled.
Calibrated probabilities, published limitations, and a visible abstention behaviour are not caveats bolted on at the end. They are what make a prediction reviewable by an expert. A number you can interrogate is worth more than a bigger number you can't.
← All notes