DISSERTATION DEFENSE

Author : Shuge Lei
Advisors: Dr. Yan Tong, Dr. Jijun Tang
Date: Nov 3, 2026
Time: 08:00 am EST/9:00pm Beijing
Location: Virtual
Link: https://teams.microsoft.com/meet/2589302190274837?p=74CDuN7xbCCRLeYd5Y
Meeting ID: 258 930 219 027 483 7
Passcode: Xa6pG24m

Abstract

A medical AI system that is usually right is not the same as one that knows when it might be wrong. This dissertation builds the layer that tells the two apart, one case at a time, and tests it on two very different systems: a breast-ultrasound classifier and a medical question-answering system that retrieves evidence before it answers (RAG). The story has three parts: Part 1 measures. Reliability is scored along two axes, grounding (does the evidence support the output?) and stability (does the output survive perturbation?). In imaging the two scores are built in by design and fused into one reliability score; in RAG an audit of fifteen training-free signals on two open language models finds the same two axes on its own, and shows that harmful retrieval can be flagged before an answer is delivered. Part 2 acts. The scores drive a three-action rule: answer, fall back to the model's own knowledge, or abstain. Abstention transfers across models, arbitration does not, and thresholds set on one model collapse on another, so every deployment must calibrate on its own data. Part 3 guarantees. Distribution-free risk control certifies those decisions with one recipe for both modalities, and returns three honest verdicts: it certifies cheaply in imaging, certifies RAG only at relaxed risk targets, and declines to certify RAG at strict ones, exactly where naive thresholds would have quietly broken their promise. The conclusion is deliberately modest. Today's signals are enough to support abstention and certified prediction in imaging; they are not yet enough to certify medical RAG at the risk levels medicine would ask for, and the framework is the instrument that says so.