[REVIEW] Detection and attribution of climate change: A deep learning and variational approach
acocac opened this issue · comments
Notebook Review: Issue #171
Submitting author: @ancazugo @SkirOwen @ViktorDomazetoski
Repository: https://github.com/eds-book-gallery/CI2023-RC-team2
Paper: https://doi.org/10.1017/eds.2022.17
Editor: @ampersandmcd @annefou
Reviewer: @asthanameghna @NHomer-Edi @dbhatedin
Managing EiC: @acocac
Status
Reviewer instructions & questions
Hi 👋 @asthanameghna @NHomer-Edi @dbhatedin, please carry out your review in this issue by updating the checklist below.
As a reviewer, you contribute to the technical quality of the content published by our community.
Before the review, EiC checked if the submission fits the minimum requirements.
The quality of the proposed contribution can be assessed through scientific, technical and code criteria.
The reviewer guidelines are available here: https://edsbook.org/publishing/guidelines/guidelines-reviewers.html.
Any questions/concerns please let @ampersandmcd know.
Review checklist for @asthanameghna
Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide.
Conflict of interest
- As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).
Code of conduct an peer-review principles
- I confirm that I read and will adhere to the Environmental Data Science code of conduct.
- I confirm that I read and will adhere to Open Science Peer Review Oath
General checks
- Notebook: Is the notebook file (
notebook.ipynb) part of the notebook repository? - Contribution and authorship: Does the author list seem appropriate and complete (full name, affiliation, and GitHub/ORCID handle)?
- Scope and eligibility: Does the submission contain an original and complete analysis according to the scope of EDS book?
Reproducibility
- Does the notebook run in a local environment?
- Does the notebook build and run in binder?
- Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?
Pedagogy
- Are the notebook purpose and highlights clear?
- Does the notebook demonstrate some specific data analysis or visualisation techniques?
- Is the notebook well documented, using both markdown cells and comments in code cells?
- Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
- Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?
Ethical
- Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
- Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
- Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?
Other Requirements
- All mentioned software should be formally and consistently cited wherever possible.
- Acronyms should be spelled out upon first mention.
- License conditions on images and figures must be respected (Creative Commons, etc.).
Final approval (post-review)
- Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.
Review checklist for @NHomer-Edi
Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide.
Conflict of interest
- As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).
Code of conduct an peer-review principles
- I confirm that I read and will adhere to the Environmental Data Science code of conduct.
- I confirm that I read and will adhere to Open Science Peer Review Oath
General checks
- Notebook: Is the notebook file (
notebook.ipynb) part of the notebook repository? - Contribution and authorship: Does the author list seem appropriate and complete (full name, affiliation, and GitHub/ORCID handle)?
- Scope and eligibility: Does the submission contain an original and complete analysis according to the scope of EDS book?
Reproducibility
- Does the notebook run in a local environment?
- Does the notebook build and run in binder?
- Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?
Pedagogy
- Are the notebook purpose and highlights clear?
- Does the notebook demonstrate some specific data analysis or visualisation techniques?
- Is the notebook well documented, using both markdown cells and comments in code cells?
- Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
- Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?
Ethical
- Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
- Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
- Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?
Other Requirements
- All mentioned software should be formally and consistently cited wherever possible.
- Acronyms should be spelled out upon first mention.
- License conditions on images and figures must be respected (Creative Commons, etc.).
Final approval (post-review)
- Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.
Review checklist for @dbhatedin
Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide.
Conflict of interest
- As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).
Code of conduct an peer-review principles
- I confirm that I read and will adhere to the Environmental Data Science code of conduct.
- I confirm that I read and will adhere to Open Science Peer Review Oath
General checks
- Notebook: Is the notebook file (
notebook.ipynb) part of the notebook repository? - Contribution and authorship: Does the author list seem appropriate and complete (full name, affiliation, and GitHub/ORCID handle)?
- Scope and eligibility: Does the submission contain an original and complete analysis according to the scope of EDS book?
Reproducibility
- Does the notebook run in a local environment?
- Does the notebook build and run in binder?
- Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?
Pedagogy
- Are the notebook purpose and highlights clear?
- Does the notebook demonstrate some specific data analysis or visualisation techniques?
- Is the notebook well documented, using both markdown cells and comments in code cells?
- Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
- Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?
Ethical
- Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
- Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
- Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?
Other Requirements
- All mentioned software should be formally and consistently cited wherever possible.
- Acronyms should be spelled out upon first mention.
- License conditions on images and figures must be respected (Creative Commons, etc.).
Final approval (post-review)
- Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.
Additional instructions
Reviewer general comments are welcome on this REVIEW issue or directly to the notebook repository.
If you do the latter, you will find a Pull Request titled REVIEW where you can carry out the discussion with authors through ReviewNB, a third-party plugin in GitHub for displaying and commenting Jupyter Notebooks (see further details here).
In addition to ReviewNB, we suggest to explore or run the notebook in:
- Binder (run): Click the Launch Binder button at the top level of this message.
The report below counts blank lines, comment lines, and physical lines of source code files using cloc. It was generated according to the latest commit 0b0ee44 of the review branch from the target repository.
Reviewers and authors feel free this info only for informative purposes. We will generate a similar report after the review process.
Software report (experimental):
github.com/AlDanial/cloc v 1.97 T=4.23 s (5.7 files/s, 2060.0 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Jupyter Notebook 5 0 3549 2082
Python 10 713 260 1669
YAML 7 20 23 339
Markdown 1 11 0 34
JSON 1 0 0 5
-------------------------------------------------------------------------------
SUM: 24 744 3832 4129
-------------------------------------------------------------------------------
👋 @asthanameghna @NHomer-Edi @dbhatedin we will conduct the review in this issue.
Please read through the above information and let me know if you have any questions about the review process.
Thank you 🙏
Hi @asthanameghna @NHomer-Edi @dbhatedin! How are you doing? Is everything OK with the review? Please let us know here when it's ready, then we'll suggest authors to go through each of your comments.
Hi @asthanameghna @NHomer-Edi @dbhatedin! How are you doing? Is everything OK with the review? Please let us know here when it's ready, then we'll suggest authors to go through each of your comments.
Hi @acocac, my comments are ready to be sent to the authors :)
My comment are ready too. I agree with all the comments by @NHomer-Edi. I am happy for the particiants to see the comments.
Hi reviewer team! Happy to share with the participants, my comments are here: "Its a very in-depth work with several innovative constructs in AI being applied at critical points being the main strength of this work. My comments are more towards the layout of the available materials - like finally the data is put up in GitLab (and code as well but thats there in github as well) which could be seen in data fetch sub-section and from the published paper's ending list. Would be great to have the data in someplace like Zenodo or OpenAire.eu. So the published paper becomes very important, a primary go to for anybody wanting to know the data being run in the code. It would be very helpful if we can see some comments in the code as well pointing to the charts, graphs and their labels. Currently some graphs are appearing towards the bottom of the notebook run without any comments, and one has to compare with the paper to properly understand. One last comment is to state the impact in a few lines, as to make a statement on what would be missing if such AI techniques were not used. In the conclusion section in the paper future work has been mentioned but the impact part would be good to add in some section."
@NHomer-Edi @asthanameghna @dbhatedin thanks for going through the checklist and provide feedback of the notebook repository.
@ancazugo @SkirOwen @ViktorDomazetoski please go through reviewers' comments and address their suggestions if relevant to have the updated version of the notebook. Note the specific comments are available in the PR of the notebook repository (see here).
@ancazugo @SkirOwen @ViktorDomazetoski may I ask your attention on above general comments left by reviewers? A reminder you can find their specific comments here.
Please let me know if you have any questions, I'm happy to help.
@ancazugo @SkirOwen @ViktorDomazetoski I just went through the reviewer's comments and see that a few of them are still pending. The notebook is really great and it would be really nice to publish it as soon as possible. Any plans to finalise it? I am happy to help if needed. Thanks,
@ViktorDomazetoski You did a very good job! Sorry I forgot I had to check another branch for the notebook. I went through all the comments and check the updated notebook. You implemented all the requested changes. It is great! There is a comment on plots that could be replaced by interactive plots: I think we can keep the plots as they are and think about adding interactivity in a future version of the notebook (which could be done by anyone interested by interactive visualisation).
From my side, the notebook is ready to be published. Thanks again for your hard work!
👋 @ancazugo @SkirOwen @ViktorDomazetoski
Congratulations! Your notebook submitted to 2023 CI Reproducibility Challenge is published! 🎉 We'll announce the release among EDS book social networks between today Monday 11th and tomorrow Tuesday 12th September.
Finally, big thanks to our editor: @annefou and the reviewers: @asthanameghna @NHomer-Edi @dbhatedin — your inputs were valuable to improving the quality of the submission 🙏