Music Generation Percent Not Accurate
grant opened this issue · comments
@cclauson
Training percentages are perfect. However, music generation percentages are not super accurate. Not a super big deal, but maybe you know why this is so.
They stay at 0% for awhile and then go 1, 2, 3, complete. Always 3%. Here's the last log I get from the status
endpoint:
{
"name": "hunbig",
"output": "Starting execution\n\nWed Mar 9 00:19:13 2016: Loading model from file ../blog-code/trained_configs//hundred\n\n/Users/granttimmerman/Documents/github/algo-rhythm/blog-code/model.py:372: UserWarning: Updating an `OrderedUpdates` with a non-ordered dictionary with 2+ elements could make your code non-deterministic\n\n self.walk_input: next_input\n\n/Users/granttimmerman/Documents/github/algo-rhythm/blog-code/model.py:375: UserWarning: Updating an `OrderedUpdates` with a non-ordered dictionary with 2+ elements could make your code non-deterministic\n\n updates.update({hidden:newstate for hidden, newstate, layer in zip(self.walk_hiddens, new_states, self.time_model.layers) if has_hidden(layer)})\n\nmodel._predict_step_note() called\n\nmodel._predict_step_note() called\n\nWed Mar 9 00:20:30 2016: Done loading model.\n\nWed Mar 9 00:20:30 2016: Generating music to output/generated...\n\nPERCENT: 1\n\nPERCENT: 2\n\nPERCENT: 3\n",
"progress": 3
}
I also get some errors from the server:
/Users/granttimmerman/Documents/github/algo-rhythm/blog-code/model.py:372: UserWarning: Updating an `OrderedUpdates` with a non-ordered dictionary with 2+ elements could make your code non-deterministic
self.walk_input: next_input
/Users/granttimmerman/Documents/github/algo-rhythm/blog-code/model.py:375: UserWarning: Updating an `OrderedUpdates` with a non-ordered dictionary with 2+ elements could make your code non-deterministic
updates.update({hidden:newstate for hidden, newstate, layer in zip(self.walk_hiddens, new_states, self.time_model.layers) if has_hidden(layer)})
Not sure if they're related.
Oh, sorry, I never responded to this.
IIRC, the way that the code we were depending on wasn't factored in a way that made it easy to monitor progress, so as a workaround we just computed a heuristic for total expected time, which was then used to show progress, but was indeed not very accurate. Just something that time didn't allow completing, I'm afraid :(