modify initial equilibrium catch for seasonal_with_size_comp model?
iantaylor-NOAA opened this issue · comments
The seasonal_with_size_comp model was originally developed in SS 3.24 (or earlier) and included an annual value for total initial equilibrium catch as it was not possible to enter season-specific values (data file lines here).
Now that the model has been converted to 3.30, it probably makes sense to divide the catch by 4 and distribute among seasons, which is likely what the authors of the original model would have done if that option had been available at the time. This requires adding additional initF parameters, and in order to get reasonable results, also requires removing this line setting lambda = 0 for the equilibrium catch: https://github.com/nmfs-stock-synthesis/ss-test-models/blob/main/models/seasonal_with_size_comp/control.ss#L519
It surely doesn't make sense to overthink all the details of these models in the test set, however, and it might be a good idea to instead routinely replace older models with newer ones that were developed using more recent practices.
having seasonal values for initial equilibirum catch does seem helpful, as I don't think it is available in the other models in the test set...
I wonder if we should try to come up with a more routine way of looking at and modifying our testing set? For example, should we let issues like this build up and then try to update the models shortly after a new release? Or is that not frequent enough?
In general, I support the idea of trying to improve this testing set, whether it is by modifying the models here or by replacing older models with newer ones.
The discussion in #8 reminded me to complete this change to refine the initial equilibrium catch assumptions for seasonal_with_size_comp in commit 2750f26.
I also modified the .gitignore in 9645d43 to exclude all the SS output files, which makes it easy to run the models in the git folder without having to filter which files to commit. Feel free to revert that change if this causes any problems for other tasks in this repository.
@k-doering-NOAA, here's a checklist from https://github.com/nmfs-stock-synthesis/stock-synthesis/wiki/Stock-Synthesis:-practices-for-maintainers#modifying-or-updating-the-ss-test-models-set (thank you for creating that list):
- clone the repo locally to get the original model files
- Make the model changes and run with the last "good" version of SS (default to the last release unless there is a reason not to).
- commit the new model input files and reference model output files to the repository
- Push up the changes to the main branch of ss-test-models
- run the build-no-est job manually on the main branch of the stock-synthesis to see if the main branch is still passing with the new model changes. If not, more investigation into whether this is a problem with SS or the model will be needed.
@iantaylor-NOAA thanks for making these changes! The build-no-est job on main branch is passing (see https://github.com/nmfs-stock-synthesis/stock-synthesis/actions/runs/810732540 ) so I am closing this issue.
I will open another issue based on our discussion re: how to improve the testing set