Next-generation MIMO systems: From index modulation to deep learning

Wireless communications has been evolving since the first generation (1G) of cellular networks, and the most recent fifth generation (5G) of the cellular communication technology electrifies both the academic world and the mobile market. Although the deployment process of 5G has not been completed yet, communication researchers have already started scrutinizing novel communication technologies for the sixth generation (6G) of the wireless world to overcome the drawbacks of 5G and build an outstanding wireless future. The thrilling applications expected to come into life with 6G, such as haptic technology, brain-computer interface, truly immersive virtual reality, and space travel, will introduce compelling requirements. Therefore, researchers have been developing innovative physical layer (PHY) communication solutions. This dissertation focuses on two promising PHY techniques for future multiple-input multiple-output (MIMO) systems: index modulation (IM) and deep learning (DL). IM methods have shown a vast potential for the next generation MIMO technologies by transmitting additional information bits utilizing various building blocks of a communication system. Specifically, the spatial modulation (SM) and its variants employ either transmit or receive antennas or both to convey additional information in the spatial domain. Therefore, SM brings high spectral efficiency to MIMO systems. In addition, the required radio-frequency (RF) chains decrease since only a portion of antennas are activated, which introduces energy efficiency. In Chapter 2, the system model of SM and its most popular variants, which are generalized SM (GSM), quadrature SM (QSM), and precoding-aided SM (PSM), are investigated in detail. Furthermore, brief literature on SM-aided MIMO systems is provided to enlighten the readers on the progress in this area. The first IM technique contribution is proposed in Chapter 3. SM systems experience the risk of information leakage to eavesdroppers, which requires increased PHY security. Therefore, this chapter presents a promising IM-based PHY security method, called CIOD-IM, for multiple-input single-output (MISO) systems. The proposed CIOD-IM scheme introduces transmit diversity and improves the system's reliability using the coordinate interleaved orthogonal designs (CIOD). In addition, CIOD-IM provides a satisfactory level of PHY security and achieves high spectral efficiency through a specially designed artificial noise (AN) matrix and novel indexing method, respectively. The bit error rate (BER) and ergodic secrecy rate (ESR) performance of CIOD-IM are investigated for both perfect and imperfect channel state information (CSI), considering the difficulties in obtaining the ideal CSI. It is illustrated by the conducted computer simulations that the proposed CIOD-IM technique exhibits superior performance over the benchmarks in both BER and ESR performance. As a more sophisticated IM method for MIMO systems, Chapter 4 presents the quadrature permutation matrix modulation (QPMM). The proposed QPMM transmission scheme divides the spatial bits into two permutation bit units in which the bits of each unit are modulated to a permutation matrix. These permutation matrices are utilized along with the singular values of the MIMO channel matrix for precoding the amplitude-phase modulated (APM) symbol vector's in-phase and quadrature components separately. The permutation indexing for both the in-phase and quadrature components doubles the spatial bits transmitted in QPMM compared to the conventional PMM method. Furthermore, a low-complexity detector, named conditional maximum likelihood detector (conditional MLD, C-MLD), is presented to overcome the complexity issue of the optimal joint MLD by decoding APM symbols separately. The BER performance of the QPMM scheme is examined assuming the Rayleigh fading channel and compared to the conventional PMM transmission scheme. The extensive computer simulations demonstrate that the QPMM scheme outperforms the conventional PMM method for various MIMO setups. Moreover, C-MLD achieves the same BER performance as the optimal joint MLD while introducing substantially lower complexity. PHY transmission schemes, specifically IM techniques, are always open to improvements. DL methods, which is the second focus of this thesis, take place with ever-growing popularity for these improvements. DL has proven its unprecedented success in diverse fields such as computer vision, natural language processing, and speech recognition by its strong representation ability and ease of computation. As the world moves forward to a thoroughly intelligent society with 6G wireless networks, new applications and use-cases have been emerging with stringent requirements for next-generation wireless communications. Therefore, recent studies have focused on the potential of DL approaches in satisfying these rigorous needs and overcoming the deficiencies of existing model-based techniques. The primary objective of Chapter 5 is to unveil the state-of-the-art advancements in the field of DL-based MIMO techniques to pave the way for fascinating applications of 6G. Up-to-date developments in DL-based techniques are examined, comparisons with state-of-the-art methods are provided, and a comprehensive guide for future directions is introduced. In particular, an overview of the underlying concepts of DL, along with the theoretical background of well-known DL techniques are presented. Furthermore, this chapter provides a programming example and the implementation of a DL-based MIMO system by sharing user-friendly code snippets, which might be useful for interested readers. DL-based strategies are analyzed for signal detection, channel estimation, and intelligent transmitter design of massive MIMO systems. Furthermore, DL-empowered IM methods are presented, which aims at evolving the existing IM techniques to more advanced and efficient levels. The ultimate goal is to enable more intelligent end-to-end (E2E) communications. Finally, Chapter 6 concludes the contributions of this dissertation with a DL-based GSM detector pointing to DL-empowered IM methods. The primary motivation of the proposed block successive interference cancellation (block SIC, B-SIC) detector is to overcome the trade-off between BER performance and complexity. B-SIC implements the linear block detection procedure to eliminate the strict antenna requirements of the conventional linear detectors, such as zero-forcing (ZF) and minimum mean squared error (MMSE) detectors. In addition, B-SIC successively decodes the APM symbols utilizing a DL model, called DeepEqualizer, beginning from the APM symbol experiencing the highest SNR. The BER performance and complexity of the proposed B-SIC detector are analyzed and compared to the optimal MLD. The performed computer simulations verify that B-SIC obtains the same BER performance as the optimal MLD with a significantly lower complexity, which is promising for massive MIMO systems with GSM.