CIF client parameter '--limit' doesn't limit the number of results returned

[jpferrero]

Hello,

The '--limit' parameter doesn't limit the numer of results returned when using cif client. I'm using CIFv5 with a ElasticSearch 7.

/usr/local/bin/cif --itype fqdn --confidence 2 --tags botnet --limit 5 -d -v

2020-07-07 19:34:09,295 - DEBUG - urllib3.connectionpool[230][MainThread] - Starting new HTTP connection (1): localhost:5000
2020-07-07 19:34:09,986 - DEBUG - urllib3.connectionpool[442][MainThread] - http://localhost:5000 "GET /indicators?itype=fqdn&confidence=2&limit=5&tags=botnet HTTP/1.1" 200 115264
+-------+----------+----------------------------+----------------------------------+-----+----+----------------------------+----------------------------+-------+--------------+-------------------+------------+-----------------------------+------------------------------+-------------+----------------------------------+
| tlp | group | reported_at | indicator | asn | cc | first_at | last_at | count | tags | description | confidence | rdata | provider | probability | reference |
+-------+----------+----------------------------+----------------------------------+-----+----+----------------------------+----------------------------+-------+--------------+-------------------+------------+-----------------------------+------------------------------+-------------+----------------------------------+
| amber | everyone | 2020-07-07T02:24:00.00000Z | restaaojrplinlm.de | | | | | 1 | botnet | 2020-07-07 01:14 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-07T01:24:00.00000Z | xnebnulmkngu.de | | | | | 1 | botnet | 2020-07-07 00:14 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-07T01:24:00.00000Z | ifxgfhxqlnkhsrdbmlxdy.de | | | | | 1 | botnet | 2020-07-07 00:14 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-07T01:24:00.00000Z | evphygiwubge.de | | | | | 1 | botnet | 2020-07-07 00:14 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-06T03:44:40.00000Z | girhrbfbggtsvcl2h.com | | | | | 1 | botnet | 2020-07-06 03:05 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-05T05:44:40.00000Z | sejehjtqvihbm.com | | | | | 1 | botnet | 2020-07-05 04:20 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-05T05:44:40.00000Z | qbvxkwoxtdhnaxk.com | | | | | 1 | botnet | 2020-07-05 04:20 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-05T05:44:40.00000Z | navvjqbilvdmipwdm.com | | | | | 1 | botnet | 2020-07-05 04:20 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
......

(more results are displayed)

The same occurs if I set --limit to more than 500, only 500 results are returned.

Using CURL instead of CIF client shows the same error:

curl -XGET http://localhost:5000/indicators?itype=fqdn&confidence=2&limit=5&tags=botnet

Thank you,
Jose

[jarandas]

Hi,

I have tested my configuration of CIF v5 with ElasticSearch v7 and I have obtained the same result that José indicated in the issue

/usr/local/bin/cif --itype fqdn --confidence 2 --tags botnet -d -v --limit 5|wc -l
2020-08-24 16:41:01,962 - DEBUG - urllib3.connectionpool[230][MainThread] - Starting new HTTP connection (1): localhost:5000
2020-08-24 16:41:02,467 - DEBUG - urllib3.connectionpool[442][MainThread] - http://localhost:5000 "GET /indicators?itype=fqdn&confidence=2&limit=5&tags=botnet HTTP/1.1" 200 144860
504


/usr/local/bin/cif --itype fqdn --confidence 2 --tags spam -d -v --limit 5|wc -l
2020-08-24 16:48:06,054 - DEBUG - urllib3.connectionpool[230][MainThread] - Starting new HTTP connection (1): localhost:5000
2020-08-24 16:48:06,536 - DEBUG - urllib3.connectionpool[442][MainThread] - http://localhost:5000 "GET /indicators?itype=fqdn&confidence=2&limit=5&tags=spam HTTP/1.1" 200 196348
504

I have tried to change the order in which the limit, marked with a limit of 500 by default is passed as a parameter but the same result is obtained

Thanks!!
jarandas.

[wes]

feel free to submit a PR with the fix. happy to merge it.