jhaco / SMDataTools

Stepmania data processing tools for StepCOVNet

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

possible advantages in using CSV files to store note data

jhaco opened this issue · comments

commented

Have been exploring the uses of having all the note data into one csv file. Will write notes here as I mull over whether or not this is needed now...

Currently, I'm writing to individual .txt files that contain the following data:

-- title
-- bpm
-- note_data
--- difficulties
---- note:timing data for each difficulty

so the output looks something like:

-- title
-- bpm
-- note_data
--- easy
---- note + timing data for easy
--- medium
---- note + timing data for medium
--- challenge
---- note + timing data for challenge

and the user will need to manage audio files seperately.


The .csv format I am considering is:

title | bpm | audio_filename | difficulty | note_data | timing_data

As each song has multiple difficulty levels, I want it to be represented something like:

title | bpm | audio_filename | easy | easy_note_data | easy_timing_data
title | bpm | audio_filename | medium | medium_note_data | medium_timing_data
title | bpm | audio_filename | challenge | challenge_note_data | challenge_timing_data

Not sure if csv can handle writing an array of data into a single cell though, that might be tricky. Could have the whole array as one ugly string, but will need to make sure it's being parsed correctly...


Pros:

  • In-house python csv module to directly parse/write CSV files to dictionaries
  • Easier bookkeeping; no more managing individual .txt files for each song

Cons:

  • Not needed atm since current implementation is working fine with StepCOVNet
  • Possible file bloat storing note:timing data for a large number of songs
  • Will need to redo all the data processing logic since the dict being passed by csv might be different than the one being used by my DataHandler
  • Too lazy if I have to do another refactor