1. Introduction

In wireless communication systems, the signal may propagate through a number of reflective paths when there are objects between the transmitter and receiver, especially for systems on the ground or in the atmosphere [1]. As attenuation, reflection, refraction, and diffraction of the signal can be caused in terms of this phenomenon, despite the thermal noise, normally considered as White Gaussian noise, varieties of fading can be introduced in these channels [1]. To figure out how the fading channels can impact the bit error rate (BER) of the received demodulated bits with different modulation schemes, a simulation based on MATLAB of PSK, QAM, and APSK modulation over Rayleigh fading, Rician fading, and Nakagami fading channels, with AWGN, the Additive White Gaussian Noise channel as contrast, is conducted in this research. This paper will introduce the concepts and approaches of the simulation, then demonstrate and analyze the simulation results to discover the best modulation scheme in different circumstances.

Phase shift keying (PSK) used in wireless computer networks and Bluetooth, is a modulation scheme that varies the phase of the signal to represent different information, while the amplitude and frequency of the signal are always constant. There are M distinct phases with the same intervals over 360 degrees for M-ary PSK. For example, for quadrature phase shift keying (QPSK), designed for two bits signal, one frequency carries four phases with an interval of 90 degrees as shown in Figure 1 [2].

Figure 1. Constellation diagram of QPSK.

Furthermore, APSK, amplitude and phase shift keying, which can be known as it consists of PSK and amplitude shift keying (ASK), varies both amplitude and phase. The advantage of its non-linear transmission scheme makes it widely used in digital video broadcasting systems [3]. The constellation diagram of APSK consists of multiple rings with different radii, in which each ring is equivalent to the diagram of PSK. The optimization of the arrangement of rings and signals is according to the maximization of symmetric information rate and mutual information over the channel [3]. Figure 2 illustrates an example of (4,12) 16-APSK with four possible signals on the internal ring and 12 on the external ring.

Figure 2. Constellation diagram of 16-APSK.

The Quadrature amplitude modulation (QAM) can be considered a particular case of APSK as it makes both amplitude and phase changes as well. One QAM signal is created by an in-phase component and quadrature component, shown as real and imaginary parts respectively in the constellation diagram. The two components create waves with particular amplitudes and phases, thus the QAM modulation is able to transmit signals with multiple bits effectively [4]. As a set of modulation schemes, there are several kinds of QAM such as square QAM, rectangular QAM, cross QAM, and star QAM with the corresponding shapes of the signals on the constellation diagram [5]. Figure 3 demonstrates the most common and typical square 16-QAM for a four-bits signal.

Figure 3. Constellation diagram of 16-QAM.

On one hand, in an AWGN channel, only the thermal white noise in a normal distribution with a mean of zero exists and it can be influenced by the signal noise ratio (SNR). On the other hand, the phase and amplitude of signal can be impacted significantly in fading channels, and this effect possibly cannot be avoided by simply increasing the SNR. The Rayleigh fading model, usually employed in multipath fading when no direct line of sight (LoS) exists in the path [6], is based on Rayleigh distribution which can be obtained by summing two uncorrelated Gaussian variables. It can thus effectively model the environment with multiple reflective items. Similarly, the Rician fading model is characterized by the Rician distribution, however, in this case, there is a dominant transmission path, typically the LoS path, among all transmission paths [7,8]. The last model introduced in this paper is Nakagami fading, including Nakagami-q, Nakagami-n, and Nakagami-m fading with different distributions [6]. This paper focuses on Nakagami-m fading with chi-square distribution where m is the fading factor. The Nakagami-m fading model can be applied in diverse multipath scenarios with varying m values, for instance, it can describe the Rayleigh distribution when m equals one [5,9]. It thus can be a useful model for indoor communication systems or communication systems in cities [9].

2. Simulation model

This section will explain why the mentioned modulation schemes and fading models are selected. In addition, the MATLAB code of simulation and selections of parameters will be discussed.

The M-PSK modulation is commonly used when M does not exceed eight, as the possible phases can only be in a range of o360 degrees. When the requirement of bandwidth efficiency goes larger, the M needs to be larger, different signals can thus be similar if the total energy, the amplitude of signals cannot be large enough. As a result, it is vital to introduce amplitude change at the same time to make signals easy to be distinguished for large alphabet size M. APSK and QAM are hence employed for simulations of alphabet sizes 16 and 32 (4 and 5 bits), while PSK and QAM are employed for simulations of alphabet size 4 and 8 (2 and 3 bits) in this project. As 4-PSK can be considered equivalent to 4-QAM and the small bit size makes the constellation diagram simple and clear, it will be used to display the impact of AWGN and fading channels. APSK and QAM schemes can achieve high data rates and they are applied in various fields, including data transmission, satellite, mobile radio systems, and so on [10]. As mentioned in the introduction, there are many kinds of QAM and the star QAM is one case of APSK. In this project, the square QAM for an even number of bits and rectangular QAM for an odd number of bits are simulated because they are frequently employed in multiple kinds of wireless standards like 3G, 4G, and 5G digital communication systems where the multipath fading can cause considerable impact [5]. Compared with QAM, APSK is more commonly used in complex environments like satellite communications as it can achieve higher minimum angular distance and smaller peak-to-average power rate (PAPR) [11,12]. In addition, when the number of alphabet sizes is the same, APSK requires a smaller amplitude level than QAM [11]. In contrast, the rectangular and square QAM can adapt to a wide range of environments. Therefore, the use of APSK or QAM depends on specific scenarios and comparisons of their performances will be involved.

Rician, Nakagami, and Rayleigh fading models are frequently applied to describe multipath fading. Rayleigh fading is most modeled, while Rician and Nakagami fading models can simulate different environments by varying their fading factors. In this project, most typical cases of these fading models will be simulated to explore the performance of different modulation schemes.

The simulation of this research is totally completed by MATLAB. A simulation of a communication system consists of four parts, which are a generation of signals, modulation, channel, and demodulation as well as calculations of BER, respectively. Firstly, the amount of bits ‘l’ transmitted can be obtained by the quantity of signals, called alphabet size. Then the product of l and the fifth power of ten random bits with values zero or one can be generated. The function ‘bit2int’ in MATLAB converts every l bit to a decimal integer, ten to the fifth power of symbols are thus created. Modulations and demodulations can be easily achieved by MATLAB functions ‘pskmod’ and ‘pskdemod’, ‘qammod’ and ‘qamdemod’, ‘apskmod’ and ‘apskdemod’. The PSK signals’ radii on the constellation diagram are one while the maximum amplitude of QAM signals increases when there are more bits. Besides, the signals are all gray-coded for accuracy. To balance the power of QAM and APSK, the radii set of 16-APSK and 32-APSK are ( \( \frac{\sqrt[]{18}}{4}, \frac{7\sqrt[]{18}}{12} \) ) and ( \( \frac{\sqrt[]{18}}{4}, \frac{7\sqrt[]{18}}{12},\sqrt[]{18} \) ). After signals pass the channel and are demodulated, the received integers can be converted back to bits by the function ‘bit2int’. The BER can be finally calculated by comparing the differences between transmitted and received bits. The introduced noise and fading models are applied in the channel parts. The AWGN channel can be achieved by function ‘awgn’ or by adding random Gaussian numbers on real and imaginary parts with function ‘randn’. For fading channels, the fading parameters are calculated according to the corresponding distributions. The signals are multiplied with fading parameters, then added with Gaussian noise, and finally equalized with the fading parameters. To make the fading models typical, the Rician factor is 10 and the fading factor of Nakagami-m fading is 2 for making it different from the other two kinds of fading.

3. Simulation results

The simulation results will be presented and analyzed with constellation diagrams and BER curves in this section. The relationship between the ratio of energy on one bit and the noise density (EbNo) and SNR is shown in equation1, where l is the number of bits. The impacts of different channels on signals will be demonstrated on constellation diagrams at EbNo 10 dB while the BER performances of 8-PSK and 8-QAM; 16-APSK and 16-QAM; 32-APSK and 32-QAM will be compared over EbNo -5 to 15 dB.

\( SNR (in dB) = EbNo + 10log{l} \) (1)

Figure 4 reveals the received signals on the constellation diagram over the AWGN channel. Influenced by the noise, received points locate in a circle with original signals as centers. Although most symbols are distorted, it is still easy to recognize where they should be under this EbNo. There are rare symbols that go out of the original area. In contrast for Rayleigh fading channel shown in Figure5, the distribution of received signals becomes much more random on the canvas. It is still denser around the original positions, however, lots of points go out of the original region, which produces problems in demodulation. The canvas is limited from -2 to 2 in both the real and imaginary axis. There can be also several points that go out of this area. The signals are clearly rotated and stretched more significantly in the Rayleigh fading channel.

Figure 4. QPSK over AWGN channel.

Figure 5. QPSK over Rayleigh fading channel.

The constellation diagram related to Rician fading is illustrated in Figure 6. It can be witnessed that symbols are rotated to a smaller extent than them over the Rayleigh channel and the distribution is like the AWGN one. The dominant LoS path contributes to this result as it mitigates the impact of reflective paths. In Figure 7, the signals over the Nakagami channel have similar distribution with the Rayleigh one while the probability of being around original positions is a little higher, which indicates a slightly lower BER.

Figure 6. QPSK over Rician fading channel.

Figure 7. QPSK over Nakagami-m fading channel.

The BER results (shown in Figure 8) match the conclusions from constellation diagrams to a significant extent. The BERs of signals experienced by Rayleigh and Nakgami fading are always highest, and they decrease approximately linearly while EbNo goes larger. Signals are least distorted in the AWGN channel, and the system reaches error free when Eb/No is greater than 10 dB. The Rician curve, having medium performance, has a similar variation with the AWGN one when EbNo does not exceed 3 dB, but the BER consequently drops linearly at higher EbNo.

Figure 8. BER performance of QPSK.

The BER curves of 8-PSK and 8-QAM over different channels are shown in Figure 9 and Figure 10. The performance of 8-PSK is slightly worse than it of QPSK because more bits are transmitted while the variation tendencies are the same. The 8-QAM system has a lower BER overall compared to 8-PSK. However, the results of the AWGN channel are nearly identical while the 8-QAM system has a slightly higher BER. The BERs of over Rayleigh and Nakagami fading channels keep varying linearly while QAM-modulated. Besides, the Rician fading merely introduces a few errors to the QAM system.

Figure 9. BER performance of 8-PSK.

Figure 10. BER performance of 8-QAM.

Figure 11 and Figure 12 demonstrate similar BER results of 16-QAM and 16-APSK. Signals are distorted most in AWGN channels when the EbNo is lower than 7 dB, but systems can reach error free at high EbNo. Systems that experienced Rician fading have the lowest BER in both cases. The 16-QAM has lower BER in all channels except the AWGN one.

Figure 11. BER curves of 16-QAM.

Figure 12. BER curves of 16-APSK.

When the alphabet size becomes larger, there is no significant difference in BER curves. Whereas in this case, the 32-QAM system has lower BER in all fading channels, and the AWGN contributes to more severe distortion on the APSK system as shown in Figure 13 and Figure 14.

Figure 13. BER curves of 32-QAM.

Figure 14. BER curves of 32-APSK.

To conclude, Rayleigh and Nakagami fading, along with white Gaussian noise at low SNR can lead to considerable distortion in the system of all modulation schemes. From the aspect of BER, the 8-QAM, and 16-QAM have better performance in fading channels, whereas 8-PSK and 16-APSK fit the AWGN channel. In addition, the 32-QAM performs better than the 32-APSK. PSK scheme demodulates received signals by comparing the phase and APSK compares both amplitude and phase, while QAM only makes decisions on amplitudes of imaginary and real parts [12]. The demodulation strategies contribute to the different performances. Furthermore, it could need to apply techniques like error correction coding or significantly high SNR for improving the BER performance of Rayleigh and Nakagami fading systems. A dominant LoS path with little disturbance in the Rician fading model can distinctly reduce errors. A high SNR can easily lead to fewer errors for the AWGN channel when the bit number is not large.

4. Conclusion

In this paper, simulations of various modulation schemes with AWGN and several fading models are completed in MATLAB. The Rayleigh and Nakagami fading can severely impact the signals while Rician and AWGN models do not distort signals significantly at high SNR. It is observed that the QAM has better BER performance in fading channels, and PSK and APSK can be more appropriate in AWGN channels. However, there are some limitations of this research. Firstly, there can be different fading factors for Rician and Nakagami fading for various environmental conditions. Moreover, there could be other QAM and APSK methods that fit the selected channels better. Most importantly, in each pair of comparisons, the total energy of each scheme is not strictly the same. As a result, the simulation only provides a general view of the best choices from the aspect of BER. All in all, the decision of modulation schemes in practice should depend on multiple factors and the model based on actual communication systems and environments. More fading models can be simulated with more kinds of QAM and APSK. In addition, the effect of error correction coding can be introduced to make the simulation more practical. Furthermore, the modulation of a larger number of bits signals with optimization of symbol amplitudes and phases normally applied in more and more communication systems in practice, can be simulated to achieve efficiency and accuracy simultaneously.