1: \begin{abstract}

2: This paper democratizes neural information retrieval to scenarios where large scale relevance training signals are not available.

3: We revisit the classic IR intuition that anchor-document relations approximate query-document relevance and propose a reinforcement weak supervision selection method, \ris{}, 

4: which learns to select anchor-document pairs that best weakly supervise the neural ranker (\textit{action}), using the ranking performance on a handful of relevance labels as the \textit{reward}. 

5: Iteratively, for a batch of anchor-document pairs, \ris{} back propagates the gradients through the neural ranker, gathers its NDCG reward, and optimizes the data selection network using policy gradients, until the neural ranker's performance peaks on target relevance metrics (\textit{convergence}).

6: % This is done by iterating the gradients on the data selection network and the neural ranker

7: % It iterates through anchor-document pairs and converges when the neural ranker's performance peaks on target relevance benchmarks.

8: In our experiments on three TREC benchmarks,

9: % Our experiments on three TREC benchmarks demonstrate the necessity and effectiveness of \ris{} in leveraging anchors as weak supervision.

10: neural rankers trained by \ris{}, with only publicly available anchor data, significantly outperform feature-based learning to rank methods and match the effectiveness of neural rankers trained with private commercial search logs.

11: Our analyses show that \ris{} effectively selects weak supervision signals based on the stage of the neural ranker training, and intuitively picks anchor-document pairs similar to query-document pairs.

12: \end{abstract}

13: