1: \begin{abstract}

2:     With a view on bilevel and PDE-constrained optimisation, we develop iterative estimates $\estdiff F(\thisx)$ of $F'(\thisx)$ for compositions $F \defeq J \circ S$, where $S$ is the solution mapping of the inner optimisation problem or PDE.

3:     The idea is to form a single-loop method by interweaving updates of the iterate $\thisx$ by an outer optimisation method, with updates of the estimate by single steps of standard optimisation methods and linear system solvers.

4:     When the inner methods satisfy simple tracking inequalities, the differential estimates can almost directly be employed in standard convergence proofs for general forward-backward type methods.

5:     We adapt those proofs to a general inexact setting in normed spaces, that, besides our differential estimates, also covers mismatched adjoints and unreachable optimality conditions in measure spaces.

6:     As a side product of these efforts, we provide improved convergence results for nonconvex Primal-Dual Proximal Splitting (PDPS).

7: \end{abstract}

8: