This document is meant to provide some guidance for SimBuild reviewers.
We recommend that reviewers also read the Guide for Authors to better understand how authors should frame their papers.
What is the Purpose of Scientific Review?
Congratulations! You’ve been asked to review papers written by other researchers for the SimBuild conference! Perhaps you’ve done this many times before, or it may be your first time. It’s useful to understand your role and how you will help the conference organizers. Reviewers provide two important services:
- They inform the scientific committee of the relevance, significance, and novelty of each paper. Reviews allow us to regulate what becomes part of the body of scientific literature, so that the papers you find in our online publications and at our conferences will be trustworthy, interesting, and influential.
- They give the authors feedback to help them get their message across clearly. Reviewers give feedback that helps authors improve their paper as much as they can to better connect with their audience. In some cases, we unfortunately can’t publish a paper based on the reviews it receives, but reviews can still help the authors improve their future papers.
Constructive Feedback
…and remember that the recipients of your review are humans, too. Criticism can be phrased in a kind way.
When you provide your feedback, make sure to include strengths and weaknesses. This serves to not only acknowledge to the authors that they did well on some parts, but it also enables the committee to make more confident decisions on which papers to accept.
SMART Feedback
When you provide feedback, keep the SMART principles in mind:
- Specific: What exactly should the authors address? (e.g., Instead of just writing “The background section is too vague.”, consider adding “Please provide more information on XZY topic.”)
- Measurable: Provide a measure for what changes you expect.
- Achievable: The changes you propose should be achievable.
- Relevant: The changes you propose should be within the scope of the paper/conference.
- Time-Bound: When you propose changes, keep in mind the provided timeframe. If the paper needs a very major change, does that reflect in your ratings and recommendation (accept/revision/reject)?
Consider Paper Type
The building simulation community has a range of experiences. Depending on the type of paper, you should adapt your review criteria.
For an overview on paper types, please refer to the Guide for Authors.
Guiding Questions
Your review should be thorough and cover all relevant aspects of the paper. You may use the reviewer questions from the BPACS 2022 training to guide your review.
- Does the title properly reflect the subject of the paper?
- Does the abstract provide an accessible summary?
- Do the keywords accurately reflect the content?
- Is the paper an appropriate length? Is it boring or repetitive?
- Are the key messages short, accurate and clear?
- Is it clear what the authors did (and why they did it)?
- Does the literature review explain the state of the art prior to this paper?
- Does the hypothesis or research question make sense?
- Does the method make sense as a way to test the hypothesis?
- Are the modeling assumptions and standards used appropriate?
- Do the results make sense? Is there anything that points to a mistake in calculations or setup? If any results were unexpected, is a reasonable explanation provided?
- Do the figures and tables match the text? If the authors say that a figure shows some result, is it easy to interpret the figure in the way the author did?
- If there is statistical analysis, does it make sense? Are the statistical measures appropriate to the question being asked?
- Are the conclusions supported by data from the paper?
- Are the references well-respected and peer reviewed?
Support the Committee
Please indicate your confidence and concerns about the manuscript. You may use the “Familiarity of the reviewer with the topic” field to let us know how confident you feel in your ability to review the manuscript accurately. Use the “Internal comments” field to explain concerns you have about your review’s accuracy or to make comments that shouldn’t be shown to the authors.
Further Guidance
If you would like to learn more about how to write great papers, you may enjoy our webinar series “Your Guide to SimBuild Success”, which includes a webinar on How to Review a Building Simulation Paper.
