Should we preregister experiments in computer vision?

ICCV 2019 Workshop

Instructions for the authors

The review cycle for a preregistered study consists of two stages: the proposal paper and the results paper. These stages reflect the exploratory (hypothesis generation) and confirmatory (hypothesis testing) phases of research.

Proposal paper

  • Read the tutorial/template author kit provided here and use it for your submission. It also describes the preregistration spirit and the motivations behind this workshop more at length.
  • Submit your paper anonymously to OpenReview. Differently from traditional computer vision conference submissions, the experimental section must only contain a description of experiments and protocol, and what conclusions can be drawn in different cases, without the results themselves. The deadline for this submission is 20th July.
  • Reviewers will assign scores to the paper to assess the quality and potential impact of the idea. They will also assess: (1) Are the experiments appropriate for validating the core hypothesis of the work? (2) Is the experimental protocol description sufficient to allow reproduction of the experiments? Reviews will be returned by 29th July.
  • You will then have a short rebuttal period to address the comments of the reviewers, by writing (up to) a single page response and making changes directly to your paper. The rebuttal period will last until 5th August.
  • Following the rebuttal, the reviewers finalise their assessments and area chairs will make accept/reject decisions for the submitted papers. Decisions will be sent to authors by 10th August.

Results paper

  • Authors of accepted papers carry out the experimental protocols proposed in their reviewed proposal paper. If the authors are unable to complete the experiments for any reason, they may withdraw it.
  • The results are then submitted in a second document known as the results paper (see the author kit). The results paper consists of just the “experimental results” and “conclusions” sections of standard publications. The results will be appended to the proposal paper to form the complete document. The deadline for the results paper is 26th October.
  • Present the work at the workshop! The date of the workshop is 2nd November.

Paper length

We are accepting both full and short format papers:

  • Full Format: The proposal paper should not exceed five pages (excluding references). There will be no strict limit for the results paper, but as a guideline we suggest 2-4 pages.
  • Short Format: The proposal paper should not exceed three pages (including references). The combined proposal and results paper should not exceed four pages (including references). Under the current rules (although it is possible these may change), use of the short format will allow the preregistered study to be usable as part of a CVPR 2020 submission.

Get started