vallieres / crawl-n-indexnow

Crawl a Shopify website to extract all URLs and submit them to IndexNow

Crawl n' IndexNow!

Get the goods and ship 'em to the indexes!

Crawl n' IndexNow is a simple CLI that pulls your Shopify or regular website sitemap, and submits the URLs to IndexNow to speed up the indexing process.

⚡️ Quickstart

Install the CLI:

go install github.com/vallieres/crawl-n-indexnow@latest

🎯 Commands

Submit All Shopify URLs to Index

crawl-n-indexnow shopify --domain legacygoods.co --key a1b3c34d

This will crawl the Legacy Goods Co. domain, list the found URLs from its sitemaps and submit them all to IndexNow.

Submit All Sitemap URLs to Index

crawl-n-indexnow sitemap --domain hipsterpixel.co --key a1b3c34d

This will crawl the Hipster Pixel domain, list the found URLs from its sitemap and submit them all to IndexNow.