webarchives

There are 3 repositories under webarchives topic.

• N0taN3rd / Squidwarc

  Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

  webarchivingwebarchivescrawlerhigh-fidelity-preservationchrome-headlesschromepuppeteerheadless-chromecrawlingbrowser-automation
  Language:JavaScript 164

• archivesunleashed / aut

  The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

  sparkhadoopwebarchivesanalysisapache-sparkscaladigital-humanitiespysparkdataframebig-data-analyticspython3big-datanetwork-graphingtext-extraction
  Language:Scala 133

• peterk / warcworker

  A dockerized, queued high fidelity web archiver based on Squidwarc

  preservationarchivingwebarchivingwebarchiveshigh-fidelity-preservation
  Language:Python 53
• robustlinks

  iipc / robustlinks

  Links on the web break all the time, robustify them!

  htmljavascriptlinksrobust-linkswebarchives
  Language:JavaScript 51

• archivesunleashed / warclight

  A Rails engine supporting the discovery of web archives.

  blacklightrubydiscoverywebarchive-discoverysolrrailswebarchiveswarcrails-engine
  Language:Ruby 48

• WebarchivCZ / Seeder

  Seeder - Czech webarchive curating tool and public site

  djangoarchivegovernmenttoolsczechczech-republicwebarchivewebarchivingwebarchives
  Language:Python 15

• archivesunleashed / docker-aut

  Docker image for the Archives Unleashed Toolkit

  dockerwebarchivessparkautdocker-imagearchives-unleashed
  Language:Dockerfile 12

• archivesunleashed / auk

  Rails application for the Archives Unleashed Cloud.

  railsrails-applicationarchives-unleashedarchives-unleashed-toolkitapache-sparkwebarchives
  Language:HTML 11
• raintale

  oduwsdl / raintale

  A Python utility for publishing a social media story built from archived web pages to multiple services.

  webarchivesmementossocial-mediastorytellingweb-archivessurrogates
  Language:Python 11
• aiu

  oduwsdl / aiu

  A library for interacting with web archive collections at Archive-It, Trove, Pandora, and more.

  webarchivesarchiveitmetadatametadata-extraction
  Language:Python 8

• toimik / WarcProtocol

  Parser for WARC (aka WebArchive) files

  warcwarc-fileswebarchivewebarchivingwebarchiveswarc-recordwarc-formatwarc-reader
  Language:C# 8
• hypercane

  oduwsdl / hypercane

  A toolkit for developing algorithms that sample mementos from a web archive collection.

  summarizationwebarchivesstorytellingmementosamplingclusteringfiltering
  Language:Python 5

• ukwa / ukwa-gsheets-utils

  Add-On for Google Sheets to help those working with web archives.

  google-sheetsgoogle-sheets-addonweb-archiveswebarchives
  Language:JavaScript 5

• N0taN3rd / whatDoWebPagesDo

  webarchivingwebarchiveshigh-fidelity-preservationweb-page-infoweb-archive-analysis
  Language:JavaScript 2

• archivesunleashed / aut-wizard

  analysissparkwebarchives
  Language:TypeScript 1

• httpreserve / conventoarchiver

  Repository for collecting scripts to help capture MyConvento newsroom press-releases from the MyConvento PR management suite. The README provides an analysis of the MyConvento URL architecture for users hoping to develop a solution for themselves.

  myconventoweb-archivingpr-newsroompress-releaseswebarchivesdigipresinternet-archiveinternet-archivingmy-convento
  Language:Python 1

• blochberger / exar

  Extract Safari Web Archives

  safariwebarchives
  Language:Python