Design and implementation of FPGA-based ANN activation functions in IQ-math number standard

Artificial Neural Networks (ANN) is a subfield of artificial intelligence developed by referencing the working structure of the human brain. Many features that exist in the human brain such as learning, classification, estimation, and decision making are the areas of study for ANN. ANN has important features such as learning and adaptation capability, parallel data processing, and the ability to be used in nonlinear systems. Due to these features, ANN has been widely used nowadays in many different fields such as image and sound processing, control, error detection, classification, optimization, prediction, decision making, secure communication, space technology, chaotic oscillator design, and medicine. Activation functions (AF) have been used in ANN studies to limit network output at the required finite intervals. Therefore, AF directly affects the result. For this reason, the choice of AF is very important. In the first stage of this thesis, linear AF TriBas (Triangular Basis), HardLim (Hard-Limit), HardLims (Symmetric Hard-Limit), PosLin (Positive Linear), SatLin (Saturating Linear), and SatLins (Symmetric Saturating Linear) functions have been coded in VHDL (Very High Speed Integrated Circuit Hardware Description Language) and designed in 32 bit (16I-16Q) IQ-Math number standard. Xilinx ISE Design Suite 14.7 program was used to test the designs. Xilinx's ISE Design Suite 14.7 program was used to test the designs. The simulation test results obtained from the testing process are presented. Then, all linear AF designs were synthesized for the XC7K70T-3FBG676 FPGA (Field Programmable Gate Arrays) chip of the Xilinx Kintex-7 family. After the synthesis process, the FPGA chip statistics are presented by performing the Place & Route process. When the chip statistics are examined, it is seen that the HardLim and HardLims functions consume the least chip resources. Besides, SatLin AF has been found to have the highest operating frequency as 1379.172 MHz among linear AF designs. In the second phase of this thesis, RadBas (Radial Basis), LogSig (Logarithmic Sigmoid), and TanSig (Tangent Sigmoid) AF, which are non-linear AF and widely used in ANN applications, are coded with VHDL in 32-bit (16I-16Q) IQ Math number standard. The CORDIC-LUT (COordinate Rotation DIgital Computer-Look Up Table) approach was used in the design of the ex exponential function in three non-linear AFs. The designs were tested in Xilinx ISE Design Suite 14.7 program and simulation test results were presented. Later, three non-linear AF designs were synthesized for the Xilinx Kintex-7 XC7K70T-3FBG676 FPGA chip. Then, the FPGA chip statistics were presented by performing the Place & Route process. Besides, MSE (Mean Square Error) and RMSE (Root Mean Square Error) error analyses of these three FPGA-based AF designs were implemented using numerical-based AF and the acquired results are presented. With these linear and non-linear AF designs, a library of activation functions has been created. In the third stage of the thesis, a sample of FPGA-based three-input and three-output feedforward ANN design has been implemented. There are 8 neurons in the hidden layer of the ANN design. LogSig AF is used in the hidden layer of the design and PureLin (Pure Linear) AF is used in the output layer. The sample ANN design has been coded in VHDL in 32 bit (16I-16Q) IQ-Math number standard. The design was tested by creating a testbench in VHDL in Xilinx ISE Design Suite 14.7 program. For the simulation results of the design obtained from the test process, MSE and RMSE error analyses were performed and the results were evaluated. After the test process, the design was synthesized for the Kintex-7 XC7K70T-3FBG676 FPGA chip, and FPGA chip statistics were presented after the Place & Route process.