knyar / nginx-lua-prometheus

Prometheus metric library for Nginx written in Lua

Repository from Github https://github.comknyar/nginx-lua-prometheusRepository from Github https://github.comknyar/nginx-lua-prometheus

Nginx warning. ngx.sleep(0) called without delayed events patch, this will hurt performance

alSergey opened this issue · comments

Hi, thank you very much for the excellent library. It significantly helps with collecting Prometheus metrics.

Describe the bug

I am using this library along with nginx-plus version r30-p2 (1.25.1) and nginx-plus-module-lua. After updating the library from version 0.20210206 to 0.20240525, I started receiving warnings from nginx.

[warn] ngx.sleep(0) called without delayed events patch, this will hurt performance, request: "GET /metrics HTTP/1.1"

As I understand it, the issue lies with this line, which was added in this PR. I locally tried changing the line ngx_sleep(0) to ngx_sleep(1), and the issue went away.

Actual behavior

There are warnings from nginx stating that the function ngx.sleep(0) is not recommended for use.

Expected behavior

No warnings from nginx.

I've added a section in README that provides more context on this.

This blog post describes the reasons for this optimization in a bit more detail:

This leads us to ngx.sleep, a non-blocking sleep function. In addition to sleeping for a specified amount of time, this function has another special purpose.

For example, if you have a piece of code that is doing intensive calculations, which takes a lot of time, the requests corresponding to this piece of code will keep taking up worker and CPU resources during this time, causing other requests to queue up and not get a timely response. At this point, we can intersperse ngx.sleep(0) to make this code give up control so that other requests can also be processed.

@knyar hi, is there any way to disable these warnings? Right now, all my logs are filled with these warnings from NGINX, and it’s impossible to use them. I had to rollback the library to version 0.20220127.

The number of warnings from NGINX is cluttering the logs significantly and makes it difficult to find really useful warnings.

@knyar hello again. I think I’ve found a way to properly disable these logs. Specifically, for the /metrics endpoint, you need to override the logging level. This will allow you to keep a different logging level for the rest of NGINX while increasing it specifically here.

location = /metrics {
  error_log /var/log/nginx/error.log error;
  content_by_lua_block {
    prometheus:collect();
  }
}

I think it would be good to add this to the FAQ. It might be useful for someone else as well.