This document tracks the status of major computer science conferences (currently defined as those listed in CSrankings) with respect to how they maintain anonymity during the reviewing process; it is curated by Emery Berger. For additions or updates, please post an issue or pull request on its GitHub repository.
For the purposes of this document:
-
Single-blind means that author identities and affiliations are visible to reviewers at all points during the reviewing process, while reviewer identities are hidden from authors.
-
At least partially double-blind means that author identities and affiliations are not revealed to reviewers for papers prior to the submission of initial paper reviews, and reviewer identities are hidden from authors.
-
Fully double-blind means that "double-blind to accept" is employed. That is, author identities and affiliations are not revealed to reviewers until the conclusion of the PC meeting, and only for accepted papers (including conditionally-accepted papers, pending modifications due to shepherding).
-
arXiv restricted means that submissions to arXiv or other pre-print servers are restricted for at least a limited period before and during the review process. When appropriate, there is a link to the CFP or other relevant document.
Each conference is linked to its call for papers (CFP) or similar. In cases where the CFP does not explicitly state the use of blind reviewing (or whether it is fully double-blind or not), this status has been verified directly with the program chair(s).
| Conference | At least partially double-blind? | Fully double-blind (blind to accept)? | arXiv restricted? |
|---|---|---|---|
| fully double-blind conferences | |||
| AAAI | Y | Y | |
| ACL | Y | Y | Y |
| ASE | Y | Y | N(1) |
| ASPLOS | Y | Y | N |
| CAV | Y | Y | N(2) |
| CCS | Y | Y | |
| CRYPTO | Y | Y | |
| CVPR | Y | Y | N |
| ECCV | Y | Y | N |
| EMNLP | Y | Y | Y |
| EuroCrypt | Y | Y | |
| EuroSys | Y | Y | |
| FAST | Y | Y | |
| FSE | Y | Y | N |
| HPCA | Y | Y | |
| ICML | Y | Y | |
| ICSE | Y | Y | Y(3) |
| IJCAI | Y | Y | N |
| ISCA | Y(*) | Y | N(4) |
| KDD | Y | Y | N |
| MICRO | Y | Y | N(4) |
| MobiCom | Y | Y | |
| NDSS | Y | Y | |
| NeurIPS | Y | Y | N |
| NSDI | Y | Y | |
| Oakland (IEEE S&P) | Y | Y | N |
| OOPSLA | Y | Y | N(5) |
| OSDI | Y | Y | |
| PLDI | Y | Y | N |
| SIGCOMM | Y | Y | N |
| SIGGRAPH | Y | Y | N |
| SIGMETRICS | Y | Y | N |
| SIGMOD | Y | Y | |
| SOSP | Y | Y | |
| USENIX ATC | Y | Y | N |
| USENIX Security | Y | Y | N |
| WWW | Y | Y | N(6) |
| partially double-blind conferences | |||
| CHI | Y(**) | N | |
| ICFP | Y | ||
| ISSTA | Y | N | |
| LICS | Y | N | |
| MobiSys | Y | ||
| POPL | Y | N | |
| single-blind conferences | |||
| FOCS | |||
| ICDE | |||
| ICRA | |||
| IROS | |||
| PODS | |||
| SODA | |||
| STOC | |||
| VIS | |||
| VLDB |
- (*): ISCA 2020's industrial track (not research track) papers are single-blind.
- (**): CHI's research paper track has a single meta-reviewer who knows the identities of all authors.
- (1): Authors are discouraged from posting the submitted work on arXiv or a similar site immediately before or after submitting to the conference. (Link)
- (2): "We do not discourage authors to put their submission on arXiv, but we strongly encourage authors to not put the work on arXiv around (within 1 week) or shortly after (within 1 month) the submission deadline, because potential reviewers may be subscribed to receive updates on recently posted papers."
- (3): "You are allowed to put your submission on your home page"
- (4): Information about arXiv submission must be provided on the submission form – the PC chair will make this information available to reviewers if it becomes necessary to ensure a fair review.
- (5): "You may post to mailing lists, arxiv, social media, or another publicity channel about your work, but do not mention where the paper is submitted and do not use the exact, as-submitted title in the posting."
- (6): "If available online (e.g., via arXiv) and not anonymous, their titles and abstract must be sufficiently different from the submission"
-
Reviewer bias in single- versus double-blind peer review, Tomkins, Zhang, and Heavlin. PNAS November 28, 2017 114 (48) 12708-12713; first published November 14, 2017. (a.k.a., the WSDM Experiment)
-
Effectiveness of Anonymization in Double-Blind Reviewing, Le Goues, Brun, Appel, Berger, Khurshid, and Smaragdakis; Communications of the ACM, May 2018. ArXiv link
-
Improving Publication Quality by Reducing Bias with Double-Blind Reviewing and Author Response; McKinley, ACM SIGPLAN Notices, 43(8):5--9, 2008.
-
More on Improving Reviewing Quality with Double-Blind Reviewing, External Review Committees, Author Response, and in Person Program Committee Meetings, McKinley, 2015.
-
A case for double-blind reviewing in software engineering, Brun, 2015.