jjackson37 / WebMapper

Website mapper that takes a single URL.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

URL Web - Create intial data structure

jjackson37 opened this issue · comments

All the URLs we extract will need to be stored in a large data structure.
The way I see it is similar to a linked list but with multiple dimensions.
The object will probably have two properties :

  • The actual URL string.
  • A collection of these objects that this URL connects to, i.e. the valid URLs that were found on this page.

With these properties we can tell what URLs are connected and in which directions.

I believe this may be best left to towards the end of project so that we can write this dependent on any requirements we hit.
A simpler data structure may be better whilst we are still RQing all the URLs, then we can convert this into the more complex structure that will be used as the basis for the display.