Performance evaluation with static analysis methods in source code security

Software vulnerabilities developed by software developers working actively in the IT sector, as well as security vulnerabilities are frequently seen. The level of security of the developed software is not evaluated in any standard framework. Static Code Analysis tools are used to discover possible vulnerabilities in software source codes. There are many paid and free testing tools such as Fortify, Polyspace, SonarCloud (open), Checkmarx, AppScan and AttackFlow, and these tools evaluate the software. While analyzing source codes, these tools produce evaluation results such as false positive, false negative, true positive and true negative. A false positive identifies vulnerabilities in the source code that actually do not have security problems and are found to be inadvertently present by static analysis tools. A false negative is expressed as a security vulnerability that cannot be found with static code analysis tools. During the security scan, who is scanning, who is requesting to make this scan, companies that are going to buy a scanning tool and companies that have to serve by hosting a security-free program in their live systems can be a victim, because of false negative and false positives. The main objective of this study is to analyze the performance of different static code analysis tools and their detection vulnerabilities. At the same time to ensure that the accuracy of a reproducible method to reveal the results of the objective assessment can be avoided. Only Polyspace and Fortify were included in the test tools available on the market, such as Fortify, Polyspace, SonarCloud (open), Checkmarx, AppScan and AttackFlow. In the study, the product of the profit, Fortify, and Polyspace products which is used in academic studies were examined and their performances were compared. In this study, the performance of software testing tools on the JULIET test data set was investigated. Then, the related test tools were analyzed and compared using different statistical methods. In order to shorten the time of the stages, various scripts have been created for filtering and automating. In addition, a project called SCATPA was developed in order to calculate the performance. In the study, the software tools aimed at testing the same weakness group (Common Weakness Enumeration, CWE) were compared with each other and the differences of the related tools; advantages - disadvantages have been demonstrated. As the thesis study is a first in its field, it is aimed to be a guide to the individuals and organizations who want to work in this field.