Sub-page requesting - Requesting valid URLs
jjackson37 opened this issue · comments
After the intial homepage has been parsed and we have all the valid URLs from that page then we need to also ping these URLs and find the URLs found on them and so on.
This will mean that the main process should be moved into some sort of process that can be looped upon so we can re-run all the extraction and filtering for the sub-pages.