Giters

retzkek / chiamon

Example Chia monitoring stack

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

debug file bug

JJassonn69 opened this issue · comments

commented

Since the log file is in INFO mode many logs are collected due to which after about 80MB of data a new log file is created which causes the mtail to malfunction and acquire random data for the variables.

only solution is to recompose the doc file in VSL which fixes the issue until another new log file is created and same happens again.

Probable solution: Maybe possible to increase the file size for log in chia or the mtail should automatically restart itself as the new file is created.

commented

mtail handles log file rotation (my logs rotate every 20 MB). So long as you're mounting the log directory into the container and not just the file it should work fine. Maybe it's another windows-only issue, in which case I recommend trying to run mtail natively on windows and not in a container.

commented

I do have mtail running on windows aswell but im not sure on how it would incorporate with the docker compose files. Any useful guides you could point me to set it up. I am using prometheus to monitor my progress remotely through my phone browser when im not at home or at work.
Again thank you for all the help.

commented

If you have mtail running in windows then you can just remove the mtail service from the compose file, and update the prometheus config to scrape mtail at host.docker.internal:9182

commented

I have no issues with running mtail under Linux, logrotate works fine.
Under windows when running mtail in a container and exposing the log I have the same issue, when the logrotate happens it seems to re-ingest the entire file multiple times exploding the values of the counters. I suspect this is related to how files are exposed in the windows subsytem for linux that docker desktop uses.

Running mtail directly under windows works but introduces another problem, it prevents the log to rotate...

commented

@pikolos how did you run the mtail in windows, could you tell me the steps.

commented

This might be due to the promtail configuration:

scrape_configs:
        - job_name: chia
          static_configs:
          - targets:
              - localhost
            labels:
              job: chia      
             __path__: /var/log/chia/*

Change the last line to:

             __path__: /var/log/chia/debug.log

Without that change, I would get all information added again whenever the logfile was rotated.

Under windows when running mtail in a container and exposing the log I have the same issue, when the logrotate happens it seems to re-ingest the entire file multiple times exploding the values of the counters. I suspect this is related to how files are exposed in the windows subsytem for linux that docker desktop uses.

This is what happened to me using linux docker

commented

thanks @kevinkk525 , willl try that, I am also logging with v=2 to see what will happen at a logrotate.

As for running mtail directly under windows just download the windows binary and run:
.\mtail.exe -logtostderr --progs C:\Users\<user>\Documents\mtail\config --logs C:\Users\<user>\.chia\mainnet\log\debug.log

putl .mtail config is in mtail\config and don't forget to modify the regex by adding \r add the end so it works with CRLF.

Then you just modify the prometheus.yml to point to host.docker.internal:9090 (asuming your docker host is the same as the host where you run chia/mtail.