wikipedia-crawler

There are 1 repository under wikipedia-crawler topic.

• Intelligent_Document_Finder

  Sarthakjain1206 / Intelligent_Document_Finder

  Document Search Engine Tool

  search-enginesearch-algorithmranking-algorithmindexerreverse-indexdocument-summarizationdocument-similaritylatent-dirichlet-allocationtext-summarizationspellcheckerwebcrawlerscrapyscrapy-spiderwikipedia-searchwikipedia-crawlerbm25
  Language:Python 68

• lehinevych / MediaWikiAPI

  Python wrapper for the MediaWiki API to access and parse data from Wikipedia

  python3wikipediawikipedia-apimediawiki-apiwikipedia-crawlerwikipedia-scwikipedia-scraper
  Language:Python 34

• nazaninsbr / Wikipedia-Crawler

  a crawler for Wikipedia (for now only the English pages)

  wikipediacrawlerwikipedia-crawlerpythonpython-crawler
  Language:Python 2

• Smile040501 / Search-Engine

  A search engine that takes keyword queries as input and retrieves a ranked list of relevant results as output. It scraps a few thousand pages from one of the seed Wiki pages and uses Elasticsearch for a full-text search engine.

  information-retrievalmarvelmaterial-uinodejs-serverreactscrapy-crawlersearch-engineweb-scrapingdirichlet-smoothingelasticsearchelasticsearch-clientlavenshteinmarvel-wikiokapi-bm25wikipedia-crawler
  Language:JavaScript 1

• TimurKasatkin / IR_system

  Innopolis IR 2016 course semester project IR system part

  search-engineranked-fulltext-searchesscalacrawlerclisbtwikipedia-crawlervector-space-modeltfidfinformation-retrieval
  Language:Scala 1

• adidottxt / wikipedia-crawler

  python web crawler to test theory that repeatedly clicking on the first link on ~97% of wiki pages eventually leads to the wiki page for knowledge 📡

  wikipedia-crawlerpython-web-crawlerbeautifulsoupbeautifulsoup4udacity
  Language:Python 0

• ambirpatel / Wikipedia-crawler

  Web scraping is data scraping technique used for extracting data from websites.

  webcrawlingwikipedia-crawler
  Language:Jupyter Notebook 0

• jamesponddotco / wikiextract

  [READ-ONLY] A word extractor for Wikipedia articles.

  crawlercrawlingdicewaregowikipediawikipedia-crawlerword-extraction
  Language:Go

• mayankkumar2 / wikipedia-index-scraper

  The program can map out the shortest path between 2 wikipedia pages.

  wikipediawikipedia-scraperwikipedia-crawlerwikipedia-entries
  Language:Go

• Relex12 / Wikipedia-Translate-Crawler

  A Wikipedia crawler that gives the worst translated page around an english starting using hypertext links

  bashtranslationweb-crawlerwikipediawikipedia-crawler
  Language:Shell

• WillCaton2350 / Wikipedia-WebCrawler

  Wikipedia Web Crawler written in Python and Scrapy. The ETL process involves multiple steps, extracting specific data from multiple wikipedia web pages/links using scrapy and organizing it into a structured format using scrapy items. Additionally, the extracted data is saved in JSON format for further analysis and integration into MySQL Workbench.

  jsonmysqlpythonpython-crawlerscrapy-crawlerscrapy-spiderweb-crawlerwikipedia-crawler
  Language:Python