Anyway to store the results in json or csv format?

Question

Anyway to store the results in json or csv format?

salooali opened this issue 3 years ago · comments

salooali commented 3 years ago

Thanks for this tool..!

I'm just having a porbelm in storing the results in a json or csv format.

Robin Verton · Answer 1 · Thu Dec 02 2021 22:54:17 GMT+0800 (China Standard Time)

So what problem are you referring to? You need to describe whats not working for you, otherwise I cant help.

salooali · Answer 2 · Thu Dec 02 2021 23:08:55 GMT+0800 (China Standard Time)

I've a huge list of domains and i want to detect what CMS or technologies they are using for this purpose i want to store the result for each domain in json or csv format but i didn't find any way to store the results

Robin Verton · Answer 3 · Thu Dec 02 2021 23:13:22 GMT+0800 (China Standard Time)

As you can see in the README.md (and via the -h flag), there is an option -output:

$ ./webanalyze -h
Usage of ./webanalyze:
  -apps string
        app definition file. (default "apps.json")
  -crawl int
        links to follow from the root page (default 0)
  -host string
        single host to test
  -hosts string
        filename with hosts, one host per line.
  -output string     <----------------------------------
        output format (stdout|csv|json) (default "stdout")
  -search
        searches all urls with same base domain (i.e. example.com and sub.example.com) (default true)
  -silent
        avoid printing header (default false)
  -update
        update apps file
  -worker int
        number of worker (default 4)

salooali · Answer 4 · Thu Dec 02 2021 23:24:01 GMT+0800 (China Standard Time)

Hey ofcourse I've tried this but this is not working for me i'm using the following command: webanalyze -host https://partakefoods.com/ -output csv It will only print the result in csv format on the terminal not on the separate file.I hope you get it.

…

On Thu, 2 Dec 2021, 8:13 pm Robin Verton, ***@***.***> wrote: As you can see in the README.md (and via the -h flag), there is an option -output: $ ./webanalyze -h Usage of ./webanalyze: -apps string app definition file. (default "apps.json") -crawl int links to follow from the root page (default 0) -host string single host to test -hosts string filename with hosts, one host per line. -output string <---------------------------------- output format (stdout|csv|json) (default "stdout") -search searches all urls with same base domain (i.e. example.com and sub.example.com) (default true) -silent avoid printing header (default false) -update update apps file -worker int number of worker (default 4) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#61 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALKTWAB4CNNN4LVNSGH4SMDUO6EJ5ANCNFSM5JHKIC3A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

Robin Verton · Answer 5 · Thu Dec 02 2021 23:32:11 GMT+0800 (China Standard Time)

You can redirect the output to a file then via your shell, for example:
webanalyze -host google.com -output csv > result.csv

salooali · Answer 6 · Fri Dec 03 2021 01:07:48 GMT+0800 (China Standard Time)

Ofcourse i can do this. But as i told you before I've a huge list of domains. There should be a way of storing the output in a file with respective domain in csv or json format.

…

On Thu, 2 Dec 2021, 8:32 pm Robin Verton, ***@***.***> wrote: You can redirect the output to a file then via your shell, for example: webanalyze -host google.com -output csv > result.csv — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#61 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALKTWAHNJGHC4C6KL3KV4LDUO6GQPANCNFSM5JHKIC3A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

Robin Verton · Answer 7 · Fri Dec 03 2021 03:23:43 GMT+0800 (China Standard Time)

I don't understand what the problem is with redirecting it into a file. It does not matter how huge your list is, this will work.

salooali · Answer 8 · Fri Dec 03 2021 13:35:11 GMT+0800 (China Standard Time)

It means If I've 10k websites than I've to create 10k individual files. Is this approach good to you? What if we store all the results in a single file in json format like Domain: {technologies}

…

On Fri, 3 Dec 2021, 12:23 am Robin Verton, ***@***.***> wrote: I don't understand what the problem is with redirecting it into a file. It does not matter how huge your list is, this will work. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#61 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALKTWADJS72ANYGSBUFVAFDUO7BUTANCNFSM5JHKIC3A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

Robin Verton · Answer 9 · Fri Dec 03 2021 15:43:02 GMT+0800 (China Standard Time)

You can load a list of domains to check and put all results in a single file:
webanalyze -hosts fileWithUrls.txt -output csv > fileWithResults.txt

Please have a look at the help output. I'm locking this issue now.