PermissionError when running with Persistance
MarinSL opened this issue · comments
I am running drain on multiple log files with persistence, as I want to later use the state to match newer logs.
However, after a random running time (varying from ~10 mins to ~80 mins), I get a Permission error:
...
Saving state of 1024 clusters with 85804 messages, 248088 bytes, reason: cluster_template_changed (7971)
Saving state of 1024 clusters with 86332 messages, 248052 bytes, reason: cluster_template_changed (7276)
Saving state of 1024 clusters with 86612 messages, 248088 bytes, reason: cluster_created (8046)
Saving state of 1024 clusters with 86668 messages, 248000 bytes, reason: cluster_template_changed (7996)
---------------------------------------------------------------------------
PermissionError Traceback (most recent call last)
Cell In[146], line 3
1 train_csv= 'passed.csv'
2 for f in filepaths:
----> 3 read_data(f,train_csv)
Cell In[140], line 44, in read_data(location, csv_location, drain, header)
42 # Add to drain
43 if drain:
---> 44 template_miner.add_log_message(message_cleaned)
46 for row in rows:
47 csv_writer.writerow(row[:-1])
File c:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\drain3\template_miner.py:141, in TemplateMiner.add_log_message(self, log_message)
139 snapshot_reason = self.get_snapshot_reason(change_type, cluster.cluster_id)
140 if snapshot_reason:
--> 141 self.save_state(snapshot_reason)
142 self.last_save_time = time.time()
143 self.profiler.end_section()
File c:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\drain3\template_miner.py:107, in TemplateMiner.save_state(self, snapshot_reason)
102 state = base64.b64encode(zlib.compress(state))
104 logger.info(f"Saving state of {len(self.drain.clusters)} clusters "
105 f"with {self.drain.get_total_cluster_size()} messages, {len(state)} bytes, "
...
1042 if "b" not in mode:
1043 encoding = io.text_encoding(encoding)
-> 1044 return io.open(self, mode, buffering, encoding, errors, newline)
PermissionError: [Errno 13] Permission denied: 'drain3_state.bin'
The drain state file is not empty, so it seems to be able to access it most of the time.
I set up drain this way:
config = TemplateMinerConfig()
config.load("drain3.ini")
persistence = FilePersistence("drain3_state.bin")
template_miner = TemplateMiner(persistence, config)
logger = logging.getLogger(__name__)
logging.basicConfig(stream=sys.stdout, level=logging.INFO, format='%(message)s')
I am running it using Jupyter extension for Visual Code, if it helps
UPD: Works fine if ran by opening Jupyter Notebook directly, so seems to be a Visual Code thing