Well, it didn’t take long for my once a week posting attempt went off track. I’m in the process of getting ICL surgery to correct my vision (my eyes are too bad for LASIK!), so I’ve missed a few days for that, plus personal time off.
This morning, I’m thinking about peer reviews – mainly because I have a couple of book chapters to review on my table right next to my laptop! This will be my first time serving as a peer reviewer for a chapter or article, though I have reviewed proposals for conferences a few times. So those chapter reviews, as well as some conference proposal reviews hanging out on my to-do list, have me thinking about logistics of these processes and how these things can be done better.
Logistically, these chapters were handled differently than journal articles. Each of the authors is expected to serve as a peer reviewer for other submitted chapters, instead of sending submissions off to external reviewers. I don’t know how common it is to do it this way for edited books. It makes sense in some ways – if you are submitting a chapter, you must have some expertise in the field, which makes it easier for the editors to assemble a group of willing reviewers. It has the added benefit that, if I read a chapter that connects somehow to my chapter, I can add in a reference to that chapter during revisions, making the book a more cohesive unit. I reviewed an edited book for Anthropos a while back that did that rather well – several chapter authors gave nods to other chapters or responded to them somehow.
One flaw in this scenario was that it was not clear in the CFP that all authors would be expected to serve as peer reviewers. Somehow, we read it thinking that each chapter submitted would equal X number of chapters to review, not each author would get X number of chapters to review. Why does this matter? In our case, I’m the first author and went into it planning to be available to do peer reviews. I was willing to take on that chunk of the responsibility as first author. My co-authors have much more scheduled over the summer, including a family vacation out of the country for two weeks during the time in which the editors want reviews to be completed. So both co-authors have had to reply that they could not complete the reviews within the time frame the editors requested due to prior obligations. We’ll see how that shakes out. We should have clarified before submitting, but the CFP could have been written more clearly as well.
But that also raises the issue of fairness. If every author and co-author is expected to complete X reviews, but my one chapter has 3 co-authors, and another author submitted 2 chapters as a single author (yet still reviews the same number of chapters as each of us), then we are doing significantly more work to get that one chapter published than that other author is doing to potentially get 2 chapters published. So what is a more fair and yet efficient way to get this done? Enlisting all co-authors and spreading the reviews out evenly means fewer reviews per person, which hopefully means a quicker turn-around time. Assigning the reviews only to the first author reduces the number of available reviewers, meaning that it will take longer to get done… Especially if one author submits multiple chapters (which I don’t know whether that has happened, but I’ve seen plenty of edited volumes in which that has happened), if that author then gets double the number of reviews to do, that really would be a big time commitment. Which way is better? Or is there a better method for reviews for an edited volume?
Conference reviews go much faster than chapter/article reviews, but they also bring some logistical challenges. I’ve served as a proposal reviewer for the Georgia International Conference on Information Literacy for a few years now, and it seems they are still trying to figure out the best way to handle reviews. Each year has been differently since I started. The first time I reviewed for them, in spring 2012, they used a rather cumbersome program. Reviewers logged in to this system, and it was not terrible but not super user-friendly. The next year, they sent out a super simple excel spreadsheet to use to review the proposals. Yeah, it’s excel (not everyone’s favorite), but it was all in one big spreadsheet, so everything is in one place, and then you send in the one big file after you complete it. I don’t remember if there were special instructions for naming the file to make it easier for them to keep it all straight, but from my end it was really simple.
This year, I was disappointed. First, each individual proposal was emailed to me in its own email. That meant that I got 20 separate emails over the course of a day (the first was at 11 am, the last was a bit after 5 pm). Please, please, if you are ever in a position to organize proposal reviews, do not do this! How easy would it have been for me to overlook one or two of those emails? I don’t think I did, and I took care to move them into their own special folder as they came in. But it would be easy for someone who gets a large quantity of email each day to overlook one or two, which you don’t want! Second, there are two links in each email – one to a google doc to fill out a rubric about the proposal, and one to a separate site to recommend whether the proposal should be accepted (with major or minor/no revisions) or rejected. There is a note that you must complete both parts. Really? How cumbersome is that?!?! Why not just add that as a question to the google doc? Or if you need it to be done through the digital commons site, why not just put the whole rubric in there?
I plan to provide this feedback after I submit my reviews this year. I’m dreading it because it looks like a pain in the butt, so I haven’t done it yet. The deadline is June 20, so I still have some time! But this will definitely be a factor when I decide whether to sign up to review proposals next year.
That said, I’m not writing that to complain. This year I also served on the planning committee for a new campus conference focused on pedagogy. Our professors have limited funds for conference travel, so most of our professors travel to conferences in their fields and present on their research, instead of getting to take time for professional development of their pedagogical techniques. So we inaugurated a new on campus conference focusing on that. The director of our new Center for Teaching & Learning (yes, our campus is a bit behind the times and just this past year established that) took the lead on organizing the conference, but I worked with her on a lot of the planning. Figuring out a good way to make it easy for reviewers to do the reviews, but also easy for us to compile the results is not the easiest thing in the world. And what seems easiest to me doesn’t always appeal as much to others – of course finding a good solution to that is a perpetual challenge of any kind of committee work!
So the point of this post is really to think through the logistics of reviewing and remember some ideas of what not to do in the future. There may not be a perfect solution to any of this, but there are certainly some solutions that are more cumbersome than others – so let’s at least try to not repeat those mistakes!