Simple Crawler and Indexer and Search Engine Web Application
Just open the project and right click the solution and choose nuget package restore. Wait till package restore completes.
-
Build and run the first project called Crawler. It uses its seed and downlaods the sites recursively (Breath First Search) and stores it in Data.Db and Crawler.Db file. Whenever you feel the gathered data is enough, simply close the program.
-
Build and run the second project called Indexer. You should copy Crawler.Db file from previous section here. After opening the program, It starts indexing the downloaded data and generates three files Sites.Db, TitleIndex.Db, and BodyIndex.Db.
-
Copy files generated from previous section to App_Data folder.
Enjoy.