1: \begin{abstract}

2:   We study acquisition functions for active learning (AL) for text classification. 

3:   The Expected Loss Reduction (ELR) method focuses on a Bayesian estimate of the reduction in classification error,

4:   recently updated with Mean Objective Cost of Uncertainty (MOCU).  

5:   We convert the ELR framework to estimate the increase in (strictly proper) scores like log probability or negative mean square error, 

6:   which we call Bayesian Estimate of Mean Proper Scores (BEMPS\footnote{Our implementation of BEMPS can be downloaded from \url{https://github.com/davidtw999/BEMPS}.}).

7:   We also prove convergence results borrowing techniques used with MOCU.

8:   In order to allow better experimentation with the new acquisition functions, 

9:   we develop a complementary batch AL algorithm, 

10:   which encourages diversity in the vector of expected changes in scores for unlabelled data.  

11:   % Further, we develop a pool filtering approach that provides initial screening of the unlabelled pool to further speed up AL with large pools

12:   To allow high performance text classifiers, 

13:   we combine ensembling and dynamic validation set construction on pretrained language models.  

14:   Extensive experimental evaluation then explores how these different acquisition functions 

15: %   and the implementation speedups 

16:   perform.  

17:   The results show that the use of mean square error and log probability with BEMPS yields robust acquisition functions, which consistently outperform the others tested.

18: \aside{Moreover, our implementation allows this class of estimation-based acquisition functions to be scaled to large pool sizes of 100,000.}

19: \end{abstract}

20: