TensorFlow variant
sdwalker62 opened this issue · comments
Good afternoon!
Our ML team uses drain3 to transform system logs as part of a larger classification pipeline. In this pipeline, we use a pre-trained template miner to transform all of the batched logs being passed into the classifier for training. We are currently investigating how this could be done using tf.data.Dataset.map API to keep the pipeline efficient.
To this end, we were curious if any other drain3 users could benefit from a TensorFlow variant of the TemplateMiner. We have experience with TensorFlow and drain3 and would be willing to begin work on such a project.
I think this might be a great contribution.
Can you outline the benefits you see in a TensorFlow variant? Will it be able to run on GPU?
Do you plan to design it as a standalone repo or as an addition to this repo? Will you re-use or reference the existing Drain3 code or rewrite it?
We are implementing Drain3 on Ray and Flink and it works well. They should greatly enhance the performance of drain deployments. Although I'm not sure what a deep learning framework based Drain3 will bring to the users, since it will be hard to accelerate anything as there's no matrix or tensor operations required.