||In recent years, wireless communication systems have undergone large scale proliferation. Not only real time voice applications, but also data applications such as file transfer protocol (FTP) applications have become popular over the wireless communication systems. Different applications have different QoS requirements to guarantee satisfactory user experience. Specifically, QoS refers to a broad collection of networking technologies and techniques. Elements of network performance within the scope of QoS often include throughput (rate), delay (latency), and error rate, etc. Furthermore, the wireless communication systems are expected to serve more and more users to meet the scalability requirements. Therefore, QoS optimization of multiuser wireless communication systems becomes one major challenge in the design of future wireless networks. In this thesis, we would like to shed some lights on this challenge. Specifically, we investigate QoS optimization in some important multiuser wireless communication systems manifested as: Delay Optimization in Random Access Networks: Random access network is a hot research topic due to its robustness in system performance. In particular, ALOHA is a popular example of random access protocol which has attracted a lot of research attention over the past two decades. However, most of the existing works are focusing on the analysis or optimization of the stability or throughput, yet the delay performance has been ignored completely. Therefore, we consider the delay-sensitive power and transmission threshold control design in S-ALOHA network with finite state Markov chain (FSMC) fading channels. To obtain a feasible and low complexity solution, we recast the optimization problem into two subproblems, namely the power control and the threshold control problem. For a given threshold control policy, the power control problem is decomposed into a reduced state Markov decision process (MDP) for single user so that the overall complexity is O(N J), where N and J are the buffer size and the cardinality of the channel state information (CSI) states. For the threshold control problem, we exploit some special structure of the collision channel and common feedback information to derive a low complexity solution. The delay performance of the proposed design is shown to have substantial gain relative to conventional throughput optimal approaches for S-ALOHA. Delay Optimization in Cellular Systems: It is well-known that cellular systems are interference limited and there are a lot of works to handle the inter-cell interference (ICI) in cellular systems. However, all of these works have assumed that there are infinite backlogs at the transmitter, and the control policy is only a function of channel state information (CSI). Therefore, we propose a distributive queue-aware intra-cell user scheduling and inter-cell interference (ICI) management control design for a delay-optimal cellular downlink system. By exploiting the special structure of the problem, we derive an equivalent Bellman equation to solve the formulated POMDP problem. To address the distributive requirement and the issue of dimensionality and computation complexity, we derive a distributive online stochastic learning algorithm, which only requires local QSI and local CSI at each of the M BSs. We show that the proposed learning algorithm converges almost-surely (with probability 1) and has significant gain compared with various baselines. The proposed solution only has linear complexity: O(M K). Delay Optimization in MIMO Cooperative Systems: There are a lot of works focusing on interference mitigation techniques for multiuser wireless communication systems. Coordinative MIMO and Cooperative MIMO are two special modes in the family of interference mitigation techniques. When backhaul constraint is considered, we need to support the concept of dynamic partial cooperative MIMO. Furthermore, applications have bursty arrivals and they are delay sensitive. Therefore, we first propose a novel partial cooperative MIMO (Pco-MIMO) physical layer, which allows flexible tradeoff between the partial data cooperation level and the backhaul consumption. Based on the Pco-MIMO scheme, we dynamically allocate power and transmission rate of the data streams according to the CSIT and queue state information (QSI) to minimize the average delay subject to backhaul capacity constraints and average power constraints. Exploiting some special structures in our problem, we propose an equivalent Bellman Equation to solve the CPOMDP. To facilitate decentralized implementations, we propose a decentralized online learning algorithm to estimate the potential and Lagrangian multiplier (LM) simultaneously, and establish our technical proof for almost-sure convergence of the learning algorithms. Moreover, we propose a novel partial stochastic gradient algorithm to derive the instantaneous power and rate control policy. The solution is completely decentralized, and is very robust to model variations. Finally, we show by simulations that there is significant performance gain compared with various baseline schemes. Delay, throughput and symbol error rate (SER) Optimization in Interference Channels: Interference has been a very difficult problem in wireless communications. For instance, the capacity region of two-user Gaussian interference channels has been an open problem for over 30 years. Therefore, it is quite challenging to optimize the QoS performance in the interference channels. We consider the delay, throughput and SER optimization to shed some lights on such challenges, respectively. In the first work, we consider delay minimization for interference networks with renewable energy source. We assume the transmit power of each node is a function of the local channel state, local data queue state and local energy queue state only. In turn, we consider two delay optimization formulations, namely the decentralized partially observable Markov decision process (DEC-POMDP) and partially observable stochastic game (POSG). In both cases, we derive a decentralized online learning algorithm to determine the control actions and Lagrangian multipliers (LMs) simultaneously, and establish the technical proof for almost-sure convergence of the learning algorithms. Furthermore, the two solutions are very robust to model variations. Finally, the delay performance of the proposed solutions are compared with conventional baseline schemes for interference networks and it is illustrated that substantial delay performance gain and energy savings can be achieved. In the second work, we propose a robust lattice alignment method for quasi-static MIMO interference channels with imperfect CSI at all SNR regimes, and a two-stage decoding algorithm to decode the desired signal. We derive the achievable data rate based on the proposed robust lattice alignment method, where the design is jointly formulated as a mixed integer and continuous optimization problem. Furthermore the derived solution is robust to imperfect CSI. We also design a low complex iterative optimization algorithm for our robust lattice alignment method. Numerical results verify the advantages of the proposed robust lattice alignment method compared with existing designs in the literature. In the third work, we propose a low complexity Partial Interference Alignment (IA) scheme in which we dynamically select the user set for IA so as to create a favorable interference profile for ID at each receiver. We first derive the average symbol error rate (SER) by taking into account of the non-Guassian residual interference due to discrete constellation. Using graph theory, we then devise a low complexity user set selection algorithm for the PIAID scheme, which minimizes the asymptotically tight bound for the average end-to-end SER performance. Moreover, we substantially simplify interference detection at the receiver using Semi-Definite Relaxation (SDR) techniques. It is shown that the SER performance of the proposed PIAID scheme has significant gain compared with various conventional baseline solutions.