urbanadventurer / WhatWeb

Next generation web scanner

Home Page:https://www.morningstarsecurity.com/research/whatweb

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Help Wanted]How can I store scan result to mysql?

imfht opened this issue · comments

commented

Question

Create the scan by run command.

./whatweb -i url_list.txt --log-sql=urls.sql

the urls.sql looks like:

INSERT IGNORE INTO targets (status,target) VALUES ('200','http://www.baidu.com/');
INSERT INTO request_configs (value) VALUES ('{"headers":{"User-Agent":"WhatWeb/0.4.9"}}')
INSERT INTO scans (target_id, config_id, plugin_id, version, os, string, account, model, firmware, module, filepath, certainty) VALUES ( (SELECT target_id from targets WHERE target = 'http://www.baidu.com/'),(SELECT MAX(config_id) from request_configs),(SELECT plugin_id from plugins WHERE name = 'Title'), '','','百度一下,你就知道','','','','','','' );
INSERT INTO scans (target_id, config_id, plugin_id, version, os, string, account, model, firmware, module, filepath, certainty) VALUES ( (SELECT target_id from targets WHERE target = 'http://www.baidu.com/'),(SELECT MAX(config_id) from request_configs),(SELECT plugin_id from plugins WHERE name = 'X-Powered-By'), '','','HPHP','','','','','','' );

the whole file is at http://termbin.com/4amj
Cloud someone how can i get the table struct?
I've tried:

./whatweb --log-sql-create 1.sql

the 1.sql looks like:

CREATE TABLE targets (target_id int NOT NULL AUTO_INCREMENT, target varchar(2048) NOT NULL, status varchar(10),PRIMARY KEY (target_id), UNIQUE (target, status) );
CREATE TABLE scans (scan_id int NOT NULL AUTO_INCREMENT, config_id INT NOT NULL, plugin_id INT NOT NULL, target_id INT NOT NULL, version varchar(255), os varchar(255), string varchar(1024), account varchar(1024), model varchar(1024), firmware varchar(1024), module varchar(1024), filepath varchar(1024), certainty varchar(10) ,PRIMARY KEY (scan_id));
CREATE TABLE request_configs (config_id int NOT NULL AUTO_INCREMENT, value TEXT NOT NULL, PRIMARY KEY (config_id) );
INSERT INTO plugins (name) VALUES ('Custom-Plugin');
INSERT INTO plugins (name) VALUES ('Grep');
INSERT INTO plugins (name) VALUES ('Conexant-EmWeb');
INSERT INTO plugins (name) VALUES ('Mahara');
INSERT INTO plugins (name) VALUES ('eMeeting-Online-Dating-Software');
INSERT INTO plugins (name) VALUES ('Trend-Micro');

When use this table struct, source scanlog.sql still do not working. the output like below:

mysql> source urls.sql;
Query OK, 0 rows affected, 1 warning (0.00 sec)

ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INSERT INTO scans (target_id, config_id, plugin_id, version, os, string, account' at line 2
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
ERROR 1048 (23000): Column 'config_id' cannot be null
Query OK, 0 rows affected, 1 warning (0.00 sec)

MySQL Version: 5.7.19

Hi @fiht
Can you try adding a ";" at the end of

INSERT INTO request_configs (value) VALUES ('{"headers":{"User-Agent":"WhatWeb/0.4.9"}}')

screen_shot_2018-02-10_at_12_14_10

commented

@andrericardo Thanks so much for your help!
After add a ";" in each

INSERT INTO request_configs (value) VALUES ('{"headers":{"User-Agent":"WhatWeb/0.4.9"}}')

It works well.