1: \begin{abstract}

2: In this paper, we consider the transceiver and relay design for the multiple-input multiple-output (MIMO) cellular two-way relay channel (cTWRC), where a multi-antenna base station (BS) exchanges information with multiple multi-antenna mobile stations via a multi-antenna relay station (RS). We propose a novel  two-way relaying scheme to approach the sum  capacity of the MIMO cTWRC. A key contribution of this work is a new non-linear lattice-based precoding technique to pre-compensate the inter-stream interference, so as to achieve efficient interference-free lattice decoding at the relay. We derive sufficient conditions for the proposed scheme to asymptotically achieve the sum capacity of the MIMO cTWRC in the high signal-to-noise ratio (SNR) regime. To fully exploit the potential of the proposed scheme, we also investigate the optimal power allocation  at the BS and the RS to maximize the weighted sum-rate of the MIMO cTWRC in the general SNR regime. It is shown that the problem can be formulated as a monotonic program, and a polyblock outer approximation algorithm is developed to find the globally optimal solution with guaranteed convergence. We demonstrate by numerical results that the proposed scheme significantly outperforms the existing schemes and closely approaches the sum capacity of the MIMO cTWRC in the high SNR regime.

3: \end{abstract}