Applications of artifical intelligence in the physical layer of wireless communication systems

Objective: The aim of this work is to analyze end to end block structure of communication system in terms of block error rate and reliability by employing deep learning-based models instead of classical techniques applied at the physical layer of wireless communication systems. Also, compared with traditional communication systems, it is aimed to attain similar or better performance results by decreasing complexity and delay. Material and Methods: In this work, initially, the performance of the modulation diversity technique which is proposed by using constellation diagrams obtained by deep learning based autoencoder is investigated for relay channel. The performance of modulation diversity applied system in the literature is compared with the autoencoder based modulator diversity system proposed for different modulation types in Rayleigh fading channel. The classical system in which modulation diversity is not applied is taken as a reference for both these two systems. As a second research direction, channel estimation is realized using deep learning and by comparing with classical channel estimation methods like "least squares" and "minimum mean square error" methods, it has been searched whether successful results are attained or not. Also, the proposed deep learning-based channel estimation method is applied in modulation diversity system and the performance is investigated for the imperfect channel state information. The simulation results in this part are obtained individually analyzed for different SNR and Doppler frequency values. Results: The amplitudes and phase of the symbols obtained by DL based model, namely by autoencoder applied learning, are found to be different from the signal constellation of classical system. Results show that both the proposed autoencoder based mapping diversity and the existing mapping diversity method in the literature attains significant improvements of 2dB and 3dB on average with respect to classical system in which no mapping diversity is applied. Also, it was found by simulation results that the performance of mapping diversity method in which autoencoder is used approaches the classical mapping diversity for different modulation types, a difference that changes by 0,4 dB and 0,6 dB on average. Finally, the MMSE channel estimation method is found to exhibit better performance than LS method for increasing SNR and Doppler frequency values. On the other hand, DL based channel estimation method is shown to have better performance than both MMSE and LS methods. Conclusion: Considering block error rate, while the autoencoder based QPSK system attains similar performance with classical system, autoencoder based 8-PSK and 16-QAM achieves better performance than classical system. Also, it was precipitated for different modulation types that while the proposed autoencoder based mapping diversity method has close performance with existing mapping diversity method in the literature, it exhibits better performance with respect to classical method with no mapping diversity. Finally, with the proposed DL based channel estimation method, the performances of both LS and MMSE channel estimation methods are improved.