Content analysis of un-structured local internet news websites

In today?s world data is a real power. In order to get advantage of the data power, analysis of the data is very important. Social incidents have been analyzed for more than a century. In order to understand social incidents better, data has big importance. Localized social analysis can be easily done by analyzing the local Internet content. In this study methods for analysis of the local news websites are discussed. On the other hand, a solution is introduced to overcome problems of un-structured website designs such as Turkish character set problems, non standard development techniques, unrelated contents such as advertisements and comments. An algorithm and code was developed to filter and index news website content. As a result code was implemented in a website and proved to be running.