Search engine architecture and application

Search engines are a technology that provides the information that people who want to create a bridge between people and data on the internet. In other words, it is a system where search engines save web sites on the web through bots, then index these pages and get meaningful data according to the queries sent by people and show the web pages containing the information they want. Users often do not know the working mechanisms of search engines. In this thesis, firstly the working architecture of the Crawler programs, which are the bot software used by the search engines, is explained and a detailed description is made by a parallel Web Crawler application. Then the indexing architecture of the search engines is described and a parallel indexing application is developed. Later, a search application was developed that describes how search engines have been searched in the indexing infrastructure and has a web interface.