1: \begin{abstract}

2: %

3: %In this work, we consider the problem of convergence of $Q$-learning with linear function approximation. We start by proposing a multi-Bellman operator that generalizes the regular Bellman operator. We find conditions under which the projected multi-Bellman operator is contractive, unlike the Bellman operator. We propose the multi $Q$-learning algorithm with linear function approximation. We show that multi $Q$-learning converges to the fixed-point of the projected multi-Bellman operator and that the solution obtained can be made arbitrarily good. We finnish by showcasing our findings on classic environments.

4: %

5: %chatgpt helped into this:

6: %

7: We study the convergence of $Q$-learning with linear function approximation. Our key contribution is the introduction of a novel multi-Bellman operator that extends the traditional Bellman operator. By exploring the properties of this operator, we identify conditions under which the projected multi-Bellman operator becomes contractive, providing improved fixed-point guarantees compared to the Bellman operator.

8: %

9: To leverage these insights, we propose the multi $Q$-learning algorithm with linear function approximation. We demonstrate that this algorithm converges to the fixed-point of the projected multi-Bellman operator, yielding solutions of arbitrary accuracy.

10: %

11: Finally, we validate our approach by applying it to well-known environments, showcasing the effectiveness and applicability of our findings.

12: \end{abstract}

13: