Thruk SLA Report - mod-livestatus error with big hostgroups/servicegroups

[Diogo Uchoas]

I have a group with over 1000 hosts and when I try to generate an Availability Report on Thruk using Livestatus, I get this error.

[1474757832] ERROR: [broker-master] [Livestatus] Back trace of this exception: Traceback (most recent call last):
  File "/var/lib/shinken/modules/livestatus/livestatus_obj.py", line 74, in handle_request
    return self.handle_request_and_fail(data)
  File "/var/lib/shinken/modules/livestatus/livestatus_obj.py", line 135, in handle_request_and_fail
    output, keepalive = query.process_query()
  File "/var/lib/shinken/modules/livestatus/livestatus_query.py", line 283, in process_query
    return self.response.respond()
  File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 147, in respond
    responselength = 1 + self.get_response_len() # 1 for the final '\n'
  File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 142, in get_response_len
    if isinstance(rsp, LiveStatusListResponse)
  File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 83, in total_len
    for generated_data in value:
  File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 278, in make_live_data_generator
    for value in self.make_live_data_generator2(result, columns, aliases):
  File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 224, in make_live_data_generator2
    item = next(result)
  File "/var/lib/shinken/modules/livestatus/livestatus_query.py", line 612, in _get_live_data_log
    for x in self.db.get_live_data_log():
  File "/var/lib/shinken/modules/logstore-sqlite/module.py", line 560, in get_live_data_log
    for rows in rows_gen:
  File "/var/lib/shinken/modules/logstore-sqlite/module.py", line 394, in select
    self.dbcursor.execute(cmd, values)
OperationalError: Expression tree is too large (maximum depth 1000)

[Olivier H]

It seems to be a limitation with sqlite :

http://www.sqlite.org/limits.html

"""The SQLITE_MAX_EXPR_DEPTH parameter determines the maximum expression
tree depth. If the value is 0, then no limit is enforced. The current
implementation has a default value of 1000"""
A quick workaround would be to use the mongodb_logstore.

An other way should be to optimise the query if possible.

Another "quick" way to fix it should be to recompile your sqlite library
with a value of "0"

Olivier

2016-09-25 1:00 GMT+02:00 Diogo Uchoas [e-mail]:

I have a group with over 1000 hosts and when I try to generate an
Availability Report on Thruk using Livestatus, I get this error.

[1474757832] ERROR: [broker-master] [Livestatus] Back trace of this exception: Traceback (most recent call last):
File "/var/lib/shinken/modules/livestatus/livestatus_obj.py", line 74, in handle_request
return self.handle_request_and_fail(data)
File "/var/lib/shinken/modules/livestatus/livestatus_obj.py", line 135, in handle_request_and_fail
output, keepalive = query.process_query()
File "/var/lib/shinken/modules/livestatus/livestatus_query.py", line 283, in process_query
return self.response.respond()
File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 147, in respond
responselength = 1 + self.get_response_len() # 1 for the final '\n'
File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 142, in get_response_len
if isinstance(rsp, LiveStatusListResponse)
File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 83, in total_len
for generated_data in value:
File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 278, in make_live_data_generator
for value in self.make_live_data_generator2(result, columns, aliases):
File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 224, in make_live_data_generator2
item = next(result)
File "/var/lib/shinken/modules/livestatus/livestatus_query.py", line 612, in _get_live_data_log
for x in self.db.get_live_data_log():
File "/var/lib/shinken/modules/logstore-sqlite/module.py", line 560, in get_live_data_log
for rows in rows_gen:
File "/var/lib/shinken/modules/logstore-sqlite/module.py", line 394, in select
self.dbcursor.execute(cmd, values)
OperationalError: Expression tree is too large (maximum depth 1000)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#73, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAxIsjNLmHYRpP63X25CjR9R47ePABtAks5qtat-gaJpZM4KFxM1
.

[Diogo Uchoas]

Good to know, you think it is possible to convert the sqlite data to mongodb for use with logstore_mongodb?