1: \begin{abstract}

2: % A pervasive issue in statistical hypothesis testing is that the reported $p$-values are biased downward by data ``peeking" -- the practice of %repeatedly recomputing the test statistic as more data samples are collected, 

3: % reporting only progressively extreme values of the test statistic as more data samples are collected. We develop practical mechanisms to eliminate the effect of peeking in common, general scenarios. 

4: % Our methods are essentially as aggressive as possible under the null, as we prove. % using techniques from random walk theory. 

5: % We demonstrate our claims empirically in a variety of ways, including for MMD- and nearest neighbor-based statistics, and to forecast ultimate future convergence of validation loss curves. The new mechanisms are available as a web tool to facilitate usage on these and any other provided statistics. 

6: A pervasive issue in statistical hypothesis testing is that the reported $p$-values are biased downward by data ``peeking" -- the practice of reporting only progressively extreme values of the test statistic as more data samples are collected. 

7: We develop principled %and practical 

8: mechanisms to estimate such running extrema of test statistics, which directly address the effect of peeking in some general scenarios. 

9: %Our methods are almost as aggressive as possible under the null., including for nonparametric %MMD- and nearest neighbor-based statistics, and to forecast future convergence of validation loss curves. The new mechanisms are available as a web tool to facilitate usage on these and any other provided statistics. 

10: \end{abstract}

11: