1: \begin{abstract}

2: Multi-task learning (MTL) aims at solving multiple related tasks simultaneously and has experienced rapid growth in recent years.

3: However, MTL models often suffer from performance degeneration with negative transfer due to learning several tasks simultaneously.

4: Some related work attributed the source of the problem is the conflicting gradients.

5: In this case, it is needed to select useful gradient updates for all tasks carefully.

6: To this end, we propose a novel optimization approach for MTL, named GDOD, which manipulates gradients of each task using an orthogonal basis decomposed from the span of all task gradients.

7: GDOD decomposes gradients into task-shared and task-conflict components explicitly and adopts a general update rule for avoiding interference across all task gradients. 

8: This allows guiding the update directions depending on the task-shared components.

9: Moreover, we prove the convergence of GDOD theoretically under both convex and non-convex assumptions. 

10: Experiment results on several multi-task datasets not only demonstrate the significant improvement of GDOD performed to existing MTL models but also prove that our algorithm outperforms state-of-the-art optimization methods in terms of \emph{AUC} and \emph{Logloss} metrics.

11: \end{abstract}

12: