Reproduce
zakariafik opened this issue · comments
Hello,
I cloned the repo and trying to play with it with my own data.
To do so, I followed the steps mentioned here. The tests run without any problem. After that, I no longer know what part should I do. Could you please clarify it?
Thanks a lot.
@zakariafik What are you interested in. For something more quick you can contribute a storage sink. We are interested in read data from many systems and writing predictions to popular cloud storage or caching technologies.
Lets start with storing perdictions to Redis:
@zmhassan Thank you so much for the prompt reply and for the extra explanation.
What I am struggling with is feeding my data to the model. Where should my input data reside (suppose a .log file).
Thanks.
Hi @zakariafik you will need to set the configuration file:
Note: LS_INPUT_PATH is where the system will read logs from.
Note: LS_OUTPUT_PATH is where the system will write predictions to.
STORAGE_DATASOURCE: "local.source"
STORAGE_DATASINK: "local.sink"
LS_INPUT_PATH: "validation_data/orders-500.log"
LS_OUTPUT_PATH: "validation_data/aug-3_25-results.txt"
INFER_ANOMALY_THRESHOLD: 1.3
INFER_TIME_SPAN: 1
INFER_LOOPS: 1
PARALLELISM: 6
Many thanks for pointing that out!
While my model is training, how can I monitor it using grafana and prometheus?
The setup is a bit complex. We are working on having a make file that will set all of it up. Should be out in a week or so.
@zakariafik We have published docs on how to install grafana and prometheus for monitoring the anomaly detection. https://log-anomaly-detector.readthedocs.io/en/latest/installation.html
Let us know if you have any questions.
@zmhassan Many thanks for keeping me updated. I will give it a try and let you know if I require any further information.
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
/lifecycle stale
Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
/lifecycle rotten
Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
/close
@sesheta: Closing this issue.
In response to this:
Rotten issues close after 30d of inactivity.
Reopen the issue with/reopen.
Mark the issue as fresh with/remove-lifecycle rotten./close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.