Tomlinson-Harashima Precoding in MIMO Systems: A Unified Approach to Transceiver Optimization Based on Multiplicative Schur-Convexity

This paper deals with the design of a multicarrier multiple-input multiple-output system which performs nonlinear Tomlinson-Harashima precoding at the transmitter and linear equalization at the receiver. The goal is the joint optimization of the processing matrices at both ends of the wireless link, assuming a constraint on the overall transmit power. We consider different design criteria which require the maximization/minimization of objective functions that depend on the mean square errors over the individual data streams. A unified framework for the solution of a great number of optimization problems is developed, based on the concept of multiplicative Schur-convexity. We show that many widely used design criteria can be easily accommodated in our framework. Our analysis is centered on two theorems which, in the case of cooperative processing among the different subcarriers, provide closed-form solutions to a number of optimization problems. On the other hand, when an independent processing is assumed at each subcarrier closed-form solutions may not exist. Nonetheless, exploiting the results of the two theorems, we show that for a variety of cost functions the optimization problem can be efficiently solved. Theoretical analysis and computer simulations are used to assess the performance of the proposed schemes and make comparisons with linear transceiver architectures. It turns out that nonlinear precoding provides better results than linear prefiltering when the cost function is multiplicatively Schur-convex. In contrast, when the cost function is multiplicatively Schur-concave the nonlinear scheme is equivalent to the linear one.