1: \begin{abstract}

2: Standard conformal prediction offers a marginal guarantee on~coverage,~but~for prediction sets to be truly useful, they should ideally ensure coverage {\it conditional}~on each test point. Unfortunately, it is impossible to achieve exact, distribution-free conditional coverage in finite samples. In this work, we propose an alternative~conformal prediction~algorithm that targets coverage where it matters most---in instances where a~classifier~is {\it overconfident in its incorrect predictions}. We start by dissecting miscoverage events in marginally-valid conformal prediction, and show that miscoverage rates vary based on~the~classifier's confidence and its deviation from the Bayes optimal classifier. Motivated by this insight, we develop a variant of conformal prediction that targets coverage conditional on a reduced set of two variables: the classifier's confidence in a prediction~and~a~nonparametric {\it trust score} that measures its deviation from the Bayes classifier. Empirical evaluation on multiple image datasets shows that our method generally improves conditional coverage properties compared to standard conformal prediction, including class-conditional coverage, coverage over arbitrary subgroups, and coverage~over~demographic~groups.

3: \end{abstract}

4: