Create a simple function to check if the data is complete
d-krupke opened this issue · comments
A very serious problem we had recently was that a third party, that did not use AlgBench, executed their experiments without any error reporting. If during the execution of multiple weeks the network or the NFS had some short term problems, entries went missing without anyone noticing. Only after the painful data analysis, we noticed inconsistencies. Because the framework in the background was not changed (and simply was bad), also after two further iterations, we noticed inconsistencies due to incomplete data or wrongly copied data.
To make sure, the data generated with the AlgBench-Framework is complete, it would be good to have a verify-mode, that does not executed anything but just throws an error, if it would. This could be a ReadOnly-Mode will do nothing if the data already exists but throws an exception if it does not.
In this context, it may also be useful to have a finalize() function that sets a ReadOnly-Flag in the internal JSON, which will trigger this mode.