1: \begin{abstract}

2: Properly designed precoders can significantly improve the spectral efficiency of multiple-input multiple-output (MIMO) relay systems. In this paper, we investigate joint source and relay precoding design based on the mean-square-error (MSE) criterion in

3: MIMO two-way relay systems, where two multi-antenna source nodes exchange information via a multi-antenna amplify-and-forward relay node. This problem is non-convex and its optimal solution remains unsolved.

4: Aiming to find an efficient way to solve the problem, we first decouple the primal problem into three tractable sub-problems, and then propose

5: an iterative precoding design algorithm based on alternating optimization.

6: The solution to each sub-problem is optimal and unique, thus the convergence of the iterative algorithm is guaranteed.

7: Secondly, we propose a structured precoding design to lower the computational complexity. The proposed precoding structure is able to parallelize the channels in the multiple access (MAC) phase and broadcast (BC) phase.

8: It thus reduces the precoding design to a simple power allocation problem.

9: Lastly, for the special case where only a single data stream is transmitted from each source node,

10: we present a source-antenna-selection (SAS) based precoding design algorithm.

11: This algorithm selects only one antenna for transmission from each source and thus requires lower signalling overhead.

12: Comprehensive simulation is conducted to evaluate the effectiveness of all the proposed precoding designs.

13: \end{abstract}

14: