For Peer Review Week 2024 Sunbul Akhtar, the Journal Development Editor at UCL Press, spoke with editors of History Education Research Journal. In this Q&A blog they discuss the thought processes involved in peer review, the challenges behind the scenes and AI, providing an insight into a function of academic publishing that has become a cornerstone of research integrity.

In this interview the questions are asked by Sunbul Akhtar at UCL Press and answered by:

Prof Arthur Chapman (AC), Editor-in-Chief of History Education Research Journal and Head of Department, Curriculum, Pedagogy and Assessment, IOE, UK

Prof Andreas Körber (AK), Editor of History Education Research Journal and Head of History Education Subject Group, Universität Hamburg, Germany

Asst Prof Lindsay Gibson (LG), Editor of History Education Research Journal and Faculty of Education, The University of British Columbia, Canada

Dr Gideon Boadu (GB), Editor of History Education Research Journal and Lecturer of Secondary Curriculum & Pedagogy Education, RMIT University, Australia

How do you select reviewers for each paper?

AC: Expertise in the relevant fields and subfields is key. Where this is not possible - for example, because the subfield that the article addresses is quite niche - general expertise in the overall field would be key.

I also look out for conflicts of interest so, if the reviewer and the author are from the same country, I look at if they are at the same university or have an obvious link.

AK: In a practical sense, the journal uses an online submission system which allows me to search through the database of past reviewers and authors. I use this to find someone in our database who has worked in the same field, but also not too closely on the same questions.

When a paper combines perspectives/approaches/methods, I try to find reviewers to address each of these.

LG: I also try to consider inviting reviewers who have not reviewed for HERJ in a while so we don't call on the same people to review articles. 

GB: I also search through the journal’s database for suitably aligned potential reviewers. In many cases, I do a general Google search for researchers with similar interests. I consider the subject matter expertise of potential reviewers and how it aligns with the paper. Another point I consider is the geographical location or region the paper is coming from. In many cases, selecting reviewers from the same region helps to ensure that contextual factors shaping the paper, like curricula, policy, etc, are considered in the review. Where papers are not tied to specific geographical regions, for example conceptual or theoretical papers, expertise becomes paramount.

What are the challenges with getting a good review?

AC: Availability is the big one. Colleagues in higher education are often very busy and they are often asked to do many reviews in addition to their contracted duties.

LG: Time is a major issue. It takes a long time to complete a review, and academics are not known for having huge amounts of unaccounted for time.

AC: It's rare to have a poor review - the people I approach usually take the role very seriously and give it the time and attention that the task deserves. On rare occasions, a review is too short or superficial. I would not rule out seeking a new review from a different academic in that situation.

LG: Also, sometimes reviewers do not provide authors with specific and descriptive feedback that can help them improve their article, or their comments are overly critical and do not focus on the positive things about the article.

GB: Since research methodologies and traditions are so diverse, not all reviewers have expertise in the specific methodologies used in a paper. This sometimes affects reviews to a good extent.

AC: On rare occasions reviews can be a little harsh or unkind in their language. In such cases, I would look to mediate that and counter-act it through my editorial comments to the author/s in the message I use to advise of review outcomes.

LG: It's important to remember that authors are fellow academics and our obligation is to help colleagues improve the quality of their writing and research. 

AC: The huge challenge is getting reviews completed in a timely manner. People are very busy and need a lot of reminding and cajoling.

If you could give three key points for reviewers to consider when they are writing what would you say?

AC: Focus on the criteria that you are asked to review to. Be tough but fair and considerate. The republic of letters should be tough but collegiate. 

Be as precise as possible - a concise set of observations is easy to respond to. Bullet pointing the key issues you wish to see addressed at the end of a review can be very helpful.

Suggest courses of action – if you feel that key literature is not consulted, for example, provide some pointers.

AK: I would say, encourage the reviewers not to be too narrow as to how they would have written the article themselves or to it having been written. Reviews are not about aligning but about a kind of broad gatekeeping.

Specifically, it is not about whether you agree to all points - something must still be left open for community to critically engage with after publication – but as  to whether the article with its specific position, perspective, methods and results is "fit to print" and to enter the discourse.

In some cases, reviewers might even highlight points they do not readily agree with, not in order to have them corrected but to point out that they might be especially valuable for further discussion.

LG: Focus on the review criteria, not your personal preferences. Write the review that you would want to receive as an author. 

GB: Offer constructive feedback that helps the authors to improve on their paperwork.

It is peer-review. Consider how you’d expect your own work to be reviewed by a peer.

A paper cannot be perfect. You are contributing to improving what has been submitted to a journal for consideration.

How would you encourage early career researchers to get involved in the peer review process?

AC: Early career researchers have so much to offer and often make the best reviewers, because they can be the most conscientious reviewers.

Reviewing is great for writing – it helps one understand how one's own papers will be judged. It's a little like having a viva on paper. Identify journals that deal with your areas of expertise, and then write to the editor/s offering your services and specifying the areas you feel competent to review. I would predict an enthusiastic response from editors – as I've said, getting the reviewers we need is a big challenge in a busy academic world. Volunteers will always be welcome in that context.

Make sure to be firm and to decline an invitation, however, if it's outside your expertise. Also, I don't think an academic need do more than 2-4 reviews a year. Don't allow editors to make too many demands on your time.

LG: Email journal editors to see if they are accepting new reviewers – most are. Only review articles you're interested and have expertise in. 

GB: Reviewing as an early career researcher helps you not only to engage with others’ work prior to publication but also build collegial relationship with editors, some of whom you are likely to ‘meet’ in the course of your career.

It prepares you for your own future research supervision.

It is a way of contributing to the field, aside from your own papers, and ensuring that the field continues to grow.

This year’s theme for peer review week is Innovation and Technology in Peer Review. With the advent of AI tools to assist writing up research, do you imagine AI could be used to provide a good review?

AK: I am doubtful. Reviewing is judging, and reviewers should not be identifiable with their clear names, but their professional expertise should be discernible. It may be that some tools may help in formulating, especially if English is not the main or first language of the reviewer, but the risk to become too formulaic may be even higher than that in the original writing of a paper, given that reviewing is a communication between concrete persons – even if anonymous.

LG: I'm also doubtful. Reviewing articles requires background knowledge, application of review criteria, and thoughtful and reasoned judgment. I'm not convinced AI is able to do this. 

Finally, do you have a message to the reviewers of the journal?

GB: Thank you for contributing your time and expertise to HERJ.

AC: I would like to thank all the colleagues who provide, and who have provided, reviews for the History Education Research Journal. A journal – and a field of research – is only as good as its peer reviewers. Peer reviewers are vital to what we do and their contribution to keeping the republic of letters running well is impossible to overstate. Thank you all for helping to ensure and to enhance the quality of history education research!

History education Peer review peer review week Q&A