The rebuttal period has ended and final decisions will be notified by January 29th. In this post, differently from part one, I will provide updated information about the decision process. For those who just want the data right now, here is the browsable table with all the submissions under review:

I left it ordered by date, and I will update the data on a daily basis so that you can check the latest updates. The final decisions will also be provided. For those who want to play with the data on their own, here it is ;). Please note that paper titles are already placed as stripped links to prevent your device from doing too much work with javascript (it might be already busy mining cryptocurrencies..).

I would also like to show you some statistics about the review-rebuttal process. First, let’s take a quick look back to the reviews:

The red line shows the daily number of reviewed submissions, and the blue line shows the accumulated amount. Please note that the deadline was the 27th of November, with ~70% of the papers reviewed. As it can be seen, the 100% is never reached. I found the cause is Causal Generative Neural Networks, which was dismissed due to the Dual Submission Policy.

The results presented in the review dates figure are somehow expected, namely a peak of reviews on the deadline. How about the behavior of the reviewers during the review period?

As it can be seen, the plot shows the score histograms during the review period. Each column represents a day, and scores are divided in three groups (colors): red (scores 0-3), blue (scores 4-6), and green (scores 7-10). It is interesting to see how extreme scores are more frequent during the first days (especially for the red group), decreasing until disappearing. It might mean that reviewers are fast to spot clear accept/reject papers, while they take more time to decide on borderline papers.

The rebuttal period finished the 5th of January. Here is the data:

In this case, a 86% of the reviews are answered by the time of the deadline, see that nice peak :)

The number of withdrawals has also increased during January, see the following figure:

The median score of the withdrawn papers is 4. Browsing in OpenReview, there can be seen some clear rejections due to fatal flaws discovered by the reviewers and admitted by the authors. This highlights the importance of reviewing for good research.

I hope you enjoyed the ICLR overview posts, and good luck with the final decisions!!

Acknowledgements

Thanks to @pepgonfaus for reviewing and encouraging me to write this series of posts.

Go to part 1, 3