Performance analysis of non-orthogonal multiple access-based 5G and beyond communication systems

In this thesis, the performance analysis of non-orthogonal multiple access (NOMA) technique with next generation communication systems is studied. At the beginning of the thesis, the performance of relay selection algorithms in amplify-and-forward (AF) and decode-and-forward (DF) relay-assisted NOMA systems is addressed in terms of bit error rate (BER). Simulation results are extensively examined to evaluate maximum-minimum (max-min) and two-stage relay selection solutions for the proposed system model. While increasing the number of relays improves BER performance, increasing the distance between the far user and the relays degrades BER performance. It is observed that the AF-assisted max-min relay selection method exhibits superior performance, especially for the far user. The BER results of the DF-assisted two-stage relay selection strategy, on the other hand, demonstrate poor performance. AF relay transmission demonstrates more effective performance for both users compared to DF relay transmission, yet a decrease in BER performance is observed as the far user moves away. In this case, AF relay transmission proves to be more effective for both users compared to DF relay transmission. Moreover, the AF assisted two stage relay selection method performs almost 10 dB gain compared to the DF scenario for the far user. In the subsequent sections of the thesis, unmanned aerial vehicle (UAV) AF-assisted relays and a downlink NOMA architecture consisting of far and near users have been investigated. The application of partial relay selection (PRS) technique based on partial channel quality for the selection of UAV relays allows for low application complexity. The best-selected UAV relay has the strongest channel gain. In some cases, access to the best UAV relay may not be possible due to factors such as load balancing, timing, or feedback delays. In this situation, alternatives such as the second, third, or n-th best UAV relays have been evaluated. The closed-form expressions for the selection of the n-th best UAV relay for bit error probability (BEP) analysis of NOMA users under a Rayleigh fading channel have been examined in detail. Additionally, diversity gain is evaluated, and asymptotic closed-form expressions for BEP under high signal-to-noise ratio (SNR) conditions are derived. Numerical results are presented to validate the accuracy of BER analytical study taking into consideration some parameters such as the number of UAV relays, path loss exponent, power allocation coefficient, and channel gains. In the rest of the thesis, an analysis of the BER performance of orthogonal time-frequency space (OTFS)-NOMA communication with high-speed movement scenarios is conducted. During BER simulations, parameters such as the speed of moving users, power allocation (PA) coefficients, delays, and Doppler effects are considered. In the proposed OTFS-NOMA system, a message passing (MP) detection algorithm is used to analyze the BER performance of users. The reflections and delays due to high-speed users are observed to impact system performance negatively. BER performances are extensively examined for 4-QAM/16-QAM modulation schemes. The BER performance of both users is observed to be negatively affected by the Doppler effect. The deep learning (DL) based OTFS-NOMA structure is integrated into next-generation communication systems within the scope of this thesis. For the proposed bidirectional long short-term memory (BiLSTM) network applied to the power-domain downlink OTFS-NOMA system model, signal detection is performed using a Rayleigh fading channel. The performance of the MP algorithm and BiLSTM technique in detecting signals from both near and far users is compared based on the symbol error rate (SER) performance criterion. The effects of commonly used stochastic gradient descent (SGD), adaptive moment estimation (ADAM), and root mean square propagation (RMSprop) optimization algorithms on SER performance are examined. According to simulation results, the BiLSTM technique outperforms MP performance for the far user. Furthermore, RMSprop is determined to outperform other optimization algorithms in terms of SER. The performance of the BiLSTM technique is observed to be more effective than the MP algorithm, except for the SGD optimization algorithm.