splunk_monitor task fails due to pagination

[Chance]

The task Fetch distributed peers when cm is defined fails in some instances due to pagination. The task retrieves a list of distributed search peers via the /services/search/distributed/peers endpoint and compares it to the list of cluster master peers retrieved in the step fetch clusterMaster peers. More specifically, the task checks to see if the first item in the cm peer list exists in the list of distributed search peers. Neither of the API calls made to retrieve these lists account for pagination and only receive the default first 30 results. If the number of peers in the indexer cluster is greater than 30 this can lead to unexpected results. One possible fix is to provide count=0 in the query parameters to receive all items. e.x. url: "{{ cert_prefix }}://127.0.0.1:{{ splunk.svc_port }}/services/search/distributed/peers?output_mode=json&count=0"
https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTprolog#Request_and_response_details
https://github.com/splunk/splunk-ansible/blob/develop/roles/splunk_monitor/tasks/main.yml#L67

[Jeff Rybczynski]

Thanks for raising the issue. It looks like we do not handle paging for these monitor (or any other) REST calls in splunk-ansible. The monitor calls seem to be the only ones that this is an issue with as the others are mostly for server status. Adding the count=0 param looks like the best fix, especially since this check really is a spot check to see that one of the peers is in the list. They all should come in at the same time so once we have a match we should be ok.