Data.write_data() saves CSV output in the same format as Data.read_data() returns -- not the original CSV format.
ncdulo opened this issue · comments
The data saved back to disk in our Data.write_data() method is saved in exactly the format we read using Data.read_data(). This seems like intended behaviour and isn't a problem with the library, more as a problem with our implementation.
I would like the data to be written back in the same format it was read. However when we read the data, we apply our own arguments which modify the format of our data in memory. Thus when writing that data back, the format has changed.
Possible fixes may include reading the data verbatim, then making a copy of it to format before display. It may also be possible to write the data back in a specified format. The latter may not be as flexible of a solution and more prone to error.
Commit eeebb85 seems to fix this issue, for now.
As mentioned in the commit description, this is likely a temporary fix. If we read the data with other arguments, transform it or sort it, we may have a modified data object write back to disk. This may not always be desired behavior depending on the context. For now though, the fix seems to hold and we should be able to move forward with implementing other features that depend on read/write capability.
Will do some further monitoring and testing over the coming days before marking this closed.
I have not encountered any issues with different input vs output in the testing since commit eeebb85 Still need to implement a unit test to ease this check, but for now it can be marked closed. Re-open or create a new issue if necessary.