[REVIEW] A Sensitivity Analysis of a Regression Model of Ocean Temperature
acocac opened this issue · comments
Notebook Review: Issue #170
Submitting author: @garimamalhotra @jedpe @dapivei
Repository: https://github.com/eds-book-gallery/repro-challenge-team-3
Paper: https://doi.org/10.1017/eds.2022.10
Editor: @annefou
Reviewer: @oscarbau @RachelFurner @ricardobarroslourenco
Managing EiC: @acocac
Status
Reviewer instructions & questions
Hi @christopheredsall @oscarbau @RachelFurner @ricardobarroslourenco, please carry out your review in this issue by updating the checklist below.
As a reviewer, you contribute to the technical quality of the content published by our community.
Before the review, EiC checked if the submission fits the minimum requirements.
The quality of the proposed contribution can be assessed through scientific, technical and code criteria.
The reviewer guidelines are available here: https://edsbook.org/publishing/guidelines/guidelines-reviewers.html.
Any questions/concerns please let @annefou know.
Review checklist for @christopheredsall (replaced by @ricardobarroslourenco)
Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide.
Conflict of interest
- As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).
Code of conduct an peer-review principles
- I confirm that I read and will adhere to the Environmental Data Science code of conduct.
- I confirm that I read and will adhere to Open Science Peer Review Oath
General checks
- Notebook: Is the notebook file (
notebook.ipynb) part of the notebook repository? - Contribution and authorship: Does the author list seem appropriate and complete (full name, affiliation, and GitHub/ORCID handle)?
- Scope and eligibility: Does the submission contain an original and complete analysis according to the scope of EDS book?
Reproducibility
- Does the notebook run in a local environment?
- #181
- Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?
Pedagogy
- Are the notebook purpose and highlights clear?
- Does the notebook demonstrate some specific data analysis or visualisation techniques?
- Is the notebook well documented, using both markdown cells and comments in code cells?
- Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
- Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?
Ethical
- Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
- Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
- Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?
Other Requirements
- All mentioned software should be formally and consistently cited wherever possible.
- Acronyms should be spelled out upon first mention.
- License conditions on images and figures must be respected (Creative Commons, etc.).
Final approval (post-review)
- Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.~~
Review checklist for @oscarbau
Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide.
Conflict of interest
- As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).
Code of conduct an peer-review principles
- I confirm that I read and will adhere to the Environmental Data Science code of conduct.
- I confirm that I read and will adhere to Open Science Peer Review Oath
General checks
- Notebook: Is the notebook file (
notebook.ipynb) part of the notebook repository? - Contribution and authorship: Does the author list seem appropriate and complete (full name, affiliation, and GitHub/ORCID handle)?
- Scope and eligibility: Does the submission contain an original and complete analysis according to the scope of EDS book?
Reproducibility
- Does the notebook run in a local environment? See comment #9
- Does the notebook build and run in binder?
- Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?
Pedagogy
- Are the notebook purpose and highlights clear?
- Does the notebook demonstrate some specific data analysis or visualisation techniques?
- Is the notebook well documented, using both markdown cells and comments in code cells?
- Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
- Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?
Ethical
- Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
- Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
- Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?
Other Requirements
- All mentioned software should be formally and consistently cited wherever possible.
- Acronyms should be spelled out upon first mention.
- License conditions on images and figures must be respected (Creative Commons, etc.).
Final approval (post-review)
- Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.
Review checklist for @RachelFurner
Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide.
Conflict of interest
- As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).
Code of conduct an peer-review principles
- I confirm that I read and will adhere to the Environmental Data Science code of conduct.
- I confirm that I read and will adhere to Open Science Peer Review Oath
General checks
- Notebook: Is the notebook file (
notebook.ipynb) part of the notebook repository? - Contribution and authorship: Does the author list seem appropriate and complete (full name, affiliation, and GitHub/ORCID handle)?
- Scope and eligibility: Does the submission contain an original and complete analysis according to the scope of EDS book?
Reproducibility
- Does the notebook run in a local environment?
- Does the notebook build and run in binder?
- Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?
Pedagogy
- Are the notebook purpose and highlights clear?
- Does the notebook demonstrate some specific data analysis or visualisation techniques?
- Is the notebook well documented, using both markdown cells and comments in code cells?
- Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
- Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?
Ethical
- Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
- Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
- Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?
Other Requirements
- All mentioned software should be formally and consistently cited wherever possible.
- Acronyms should be spelled out upon first mention.
- License conditions on images and figures must be respected (Creative Commons, etc.).
Final approval (post-review)
- Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.
Review checklist for @ricardobarroslourenco (replacing @christopheredsall)
Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide.
Conflict of interest
- As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).
Code of conduct an peer-review principles
- I confirm that I read and will adhere to the Environmental Data Science code of conduct.
- I confirm that I read and will adhere to Open Science Peer Review Oath
General checks
- Notebook: Is the notebook file (
notebook.ipynb) part of the notebook repository? - #179
- Scope and eligibility: Does the submission contain an original and complete analysis according to the scope of EDS book?
Reproducibility
- Does the notebook run in a local environment?
- #180
- Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?
Pedagogy
- Are the notebook purpose and highlights clear?
- Does the notebook demonstrate some specific data analysis or visualisation techniques?
- Is the notebook well documented, using both markdown cells and comments in code cells?
- Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
- Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?
Ethical
- Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
- Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
- Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?
Other Requirements
- All mentioned software should be formally and consistently cited wherever possible.
- Acronyms should be spelled out upon first mention.
- License conditions on images and figures must be respected (Creative Commons, etc.).
Final approval (post-review)
- Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.
Additional instructions
Reviewer general comments are welcome on this REVIEW issue or directly to the notebook repository.
If you do the latter, you will find a Pull Request titled REVIEW where you can carry out the discussion with authors through ReviewNB, a third-party plugin in GitHub for displaying and commenting Jupyter Notebooks (see further details here).
In addition to ReviewNB, we suggest to explore or run the notebook in:
- Binder (run): Click the Launch Binder button at the top level of this message.
The report below counts blank lines, comment lines, and physical lines of source code files using cloc. It was generated according to the latest commit 1111e77 of the review branch from the target repository.
Reviewers and authors feel free this info only for informative purposes. We will generate a similar report after the review process.
Software report (experimental):
github.com/AlDanial/cloc v 1.97 T=7.84 s (3.6 files/s, 1232.5 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Python 16 731 554 4962
YAML 6 20 23 786
Text 2 9 0 413
Jupyter Notebook 1 0 1756 331
Markdown 1 11 0 34
Bourne Shell 1 6 4 19
JSON 1 0 0 5
-------------------------------------------------------------------------------
SUM: 28 777 2337 6550
-------------------------------------------------------------------------------
👋 @christopheredsall @oscarbau @RachelFurner we will conduct the review in this issue.
Please read through the above information and let me know if you have any questions about the review process.
Thank you 🙏
Hi @christopheredsall @oscarbau @RachelFurner! How are you doing? Is everything OK with the review? How is it going? Do you need any help?
@RachelFurner Thanks for all your comments on this notebook. Would you mind providing a general feedback of your review? You left some unchecked items in Pedagogy, so maybe you could elaborate on these aspects. That would be very helpful. Thanks.
Hi @christopheredsall Any updates on the review? Do you have any estimated date for your final review? Or do you have any barrier to conduct the review? Thanks.
@oscarbau Thanks for your comments and helping to improve the quality of the notebook. Same question than for Rachel. Would you mind providing a general feedback on the review of the notebook? And you also have some uncheck items in Pedagogy. Could you elaborate and explain how the notebook could be improved? Thank you!
Hi, huge apologies for the lack of finalising the review. I had a really limited amount of time available for this, with a hard deadline as I am now on maternity leave. I then struggled to get to grips with the NB review and downloading the data etc. Coupled with this, a number of other pressing work things came up, and I never managed to complete all aspects of the review. I've had a quick scan, and it seems the issues I raised have been addressed. If the other reviewers are happy with things then I am happy to defer to them on this. Apologies for not being able to be more thorough.
Hi, huge apologies for the lack of finalising the review. I had a really limited amount of time available for this, with a hard deadline as I am now on maternity leave. I then struggled to get to grips with the NB review and downloading the data etc. Coupled with this, a number of other pressing work things came up, and I never managed to complete all aspects of the review. I've had a quick scan, and it seems the issues I raised have been addressed. If the other reviewers are happy with things then I am happy to defer to them on this. Apologies for not being able to be more thorough.
Dear @RachelFurner thanks for your message. You do not need to apologize. We completely understand the limited time of reviewers and special circumstances such your maternity leave. We are really grateful for your contribution and hope the proposed notebook facilitates the understanding of your paper to future readers.
Sorry for any inconvenience with the suggested ReviewNB platform and data downloading issues. We'll consider this to future improvements of the review system.
Hi @christopheredsall since we have not heard from you in several weeks, we are now looking for a new reviewer. Thank you for your original willingness to contribute a review.
@garimamalhotra @jedpe @dapivei I'll act as a reviewer of your submission to replace @christopheredsall. Please expect some general and specific comments from me within this week.
@garimamalhotra @jedpe @dapivei I'll act as a reviewer of your submission to replace @christopheredsall. Please expect some general and specific comments from me within this week.
Ok, @ricardobarroslourenco kindly has accepted an invite to act as a reviewer in this submission. Ricardo, please 👍 if you're happy to confirm here.
Very happy to join this review ˆˆ
Hi everyone. I think that I am almost done with the review process. The only point that keeps me of giving final approval is regarding the Binder run (I have opened an issue on it), which is crashing. I my view it seems that the notebook does consumes more resources than Biner provides, so the python kernel restarts. However, I was able to run it on a HPC node, so seems to work fine for me. @acocac @annefou let me know your thoughts.
Thanks @oscarbau @RachelFurner @ricardobarroslourenco for going through the notebook and checking all the different criteria. It looks like all your comments were addressed. It would be awesome if you could go through the Final approval (post-review) box. Thanks!
@annefou we have a remaining issue (which needs evaluation if it can be done, though) originally posted at #180 and moved by @acocac to eds-book-gallery/3286b92f-4fae-4cc6-a29e-e408bc844542#14 . I believe the workflow cannot be run on Binder, because it is large, and reducing the data will likely jeopardize the experiment results.
@annefou we have a remaining issue (which needs evaluation if it can be done, though) originally posted at #180 and moved by @acocac to eds-book-gallery/repro-challenge-team-3#14 . I believe the workflow cannot be run on Binder, because it is large, and reducing the data will likely jeopardize the experiment results.
@ricardobarroslourenco yes we are aware of this issue but I had the feeling that apart from this issue the notebook fulfils the requirements. Am I wrong? Thanks.
Yes. Well, I will mark it as checked, given the circumstances (@acocac perhaps it would be nice to address this later on general instructions for further reviewers)
Hi, huge apologies for the lack of finalising the review. I had a really limited amount of time available for this, with a hard deadline as I am now on maternity leave. I then struggled to get to grips with the NB review and downloading the data etc. Coupled with this, a number of other pressing work things came up, and I never managed to complete all aspects of the review. I've had a quick scan, and it seems the issues I raised have been addressed. If the other reviewers are happy with things then I am happy to defer to them on this. Apologies for not being able to be more thorough.
@annefou I'll check @RachelFurner's Final approval (post-review) box according to her message above.
Hi @acocac, this notebook is ready to be published
Hey All, Thanks for the review. Let us know if there is anything else we need to do during the publishing process.
Hey All, Thanks for the review. Let us know if there is anything else we need to do during the publishing process.
Hi @garimamalhotra thanks for your message. @annefou and I are working on how to improve fetching the notebook input files. We are testing cloud-optimized formats such as ZARR to facilitate accessing and processing large files. This could speed-up running the notebook and pass the automated checks i.e. GitHub actions we usually have for EDS book notebooks.
Congratulations, @garimamalhotra @jedpe @dapivei, your EDS book notebook is published! 🎉 We'll announce the release among EDS book social networks on Monday 4th September. We suggest then to wait until the suggested release date. In the meantime, EDS book maintainers will be working on some key updates of the website and repository. Thanks for the understanding.
Finally, big thanks to our editor: @annefou and the reviewers: @oscarbau @RachelFurner @ricardobarroslourenco — you all contributed to improving the quality of the submission 🙏