Developing doctoral students’ critical writing skills through peer assessment and review

Tags

Critical thinking, part-time study, Peer review, student journal

Our guest blogger this week is Dr Joan Woodhouse, Associate Professor of Education at the University of Leicester UK. She has been working with doctoral students for approximately thirteen years and has a keen interest in developing early career researchers’ academic writing skills. Her other research interests include teachers’ lives and careers, and in particular, women teachers’ lives and career experiences. She is currently engaged in collaborative research into the experiences of student teachers who are mothers, with a view to considering how providers of Initial Teacher Education might better support this group of women, who she sees as the ‘invisible statistic’ in the equality monitoring data.

By Joan Woodhouse

As Programme Director for the Doctorate of Education (EdD), I have worked with numerous full-time teachers and school leaders who combine their professional work with part-time study. The EdD students are typically mid- to late-career teachers, often occupying fairly senior positions in school. As doctoral students they can be rather isolated from other, often younger, full-time and campus-based postgraduates. Their main point of contact with the University is with their supervisor. So, it can be, at best, challenging for this group of students to feel a sense of belonging to the academic and research community. Many of the EdD students are also returning to academic study after a significant gap and can feel uncertain in navigating the unwritten rules and culture of academia. They are expected to read and write critically, often without any explicit guidance on what that means. They are given feedback on their work in which the weaknesses in their arguments are highlighted, their poor expression exposed and the gaps in their knowledge indicated. It can be hard to take the critical feedback: affective barriers can impede students’ ability to assimilate and act on the feedback, yet assimilation and action are vital if they are to progress. Learning to respond to feedback in ways that are pragmatic rather than emotional is a big part of developing resilience as a researcher. I have come to realise that as supervisors and tutors we need strategies to foster this resilience in our students if they are to become the critical thinkers, readers and writers we need them to be to succeed at doctoral level.

This post is about a project in which a colleague and I inducted doctoral students into peer review by (i) involving them in peer assessment activities and (ii) supporting them to set up and run a student journal. Our aim was to facilitate the development of students’ critical writing skills, by engaging them in giving and receiving critical feedback (both of which they were fairly anxious about at the start).

Should doctoral writers do free work for academic journals?

Tags

journal article, Peer review, publication practices

By Cally Guerin

Doctoral writers are often keen to publish their work in highly ranked journals, thus entering the debate about whether or not academics should publish their work in journals run by the big publishing companies. This contention continues, especially with regard to COVID-related research. Those powerful academic publishers are accused of exploiting the voluntary work of researchers and scholars through the administrative and editorial load they undertake in organising and performing peer review, in making decisions about manuscripts, in corresponding with authors, and in finalising the published articles.

Even though I have many sympathies with the open access movement and applaud the efforts to make publicly funded research freely available to the public, I still do unpaid work for journals owned by big publishers. There’s value for me in reviewing submissions and in handling articles as an associate editor. I want to explain my reasons for doing this free work so that doctoral writers make informed decisions about what is right for them.  

Doctoral writing: Playing in woods and trees

Tags

argumentation, feedback on writing, Feedback practices, Peer review, Precision, research writing, writing fun

By Susan Carter

Being unable to see the woods for the trees is a metaphor that is sometimes applied to thesis writing for when close attention to detail (the trees) causes an author to lose oversight of the purpose and shape of the whole thesis (the woods). Thesis writers sometimes mention that they pin their research question, or their overall argument, above their desk as a pointer reminding them that when they are focusing on detail, writing should always be within the framework of the big picture.

For a two-hour doctoral writers’ workshop, I drew on the woods and trees metaphor to encourage both an overview of the big picture and attention to detail.

A life in review: Writing tasks that academics do that we don’t talk about

Tags

Academic reviewing, Peer review

This guest post is from Sue Starfield, professor in the School of Education and the Director of the Learning Centre at UNSW Australia. Sue’s interests include tertiary academic literacies, doctoral writing, writing for publication and identity in academic writing. If you enjoy this, you may like these related posts.

It struck me recently that I spend large amounts of my everyday academic life carrying out reviews of various sorts. Besides the ongoing feedback I provide to my doctoral students on their writing, usually through ‘track changes’, I do many other kinds of reviews. Quite a number of these are quite high stakes such as examining a doctoral thesis or reviewing a book proposal for a publisher for example. Yet it is difficult, if not impossible, to gain access to exemplars of these kinds of texts.

Hyland and Diani (2009, p. 1) noted that “what academics mainly do is evaluate”. As Langveldt and Kyvik (2011, p. 199) point out in their examination of the multiple roles researchers play as evaluators in the course of their academic lives, these roles often have a gatekeeping function as researchers “provide or deny access to opportunities for fellow colleagues to do research, to publish research, and to get tenure or promotion”. But if these review/evaluation genres are not publicly available, how then do doctoral students as early career academics learn to write them?

Peer review: what’s the fuss?

Tags

open access, Peer review, publishing research

By Cally Guerin

We’ve posted a few blogs on this site relating to doctoral candidates giving and receiving peer review in writing groups, and the feedback that supervisors provide on writing. Another kind of peer review that PhD students can receive is that generated by journal reviewers, which raises a different set of concerns from those we’ve discussed elsewhere.

PhD students are often encouraged to publish their research in academic journals, but it can be quite daunting to send work out to an unknown audience who will judge whether it is worthy or unworthy of publication. Everyone has stories about receiving harsh, unfair reviews when they’ve submitted their work to journals. However much we try to tell ourselves and our students that ‘it isn’t personal’, it does feel personal at the time of getting negative responses. As Kate Chanock amusingly points out, the process can feel rather like going through the stages of grief – but in this case, it’s the Seven Stages of Resentment. Of course she is being tongue in cheek, but there is more than a hint of truth in what she says here.

Despite the problems with peer review, it underpins most academic work as the usual process for assessing grant applications through to publishing the results of those grants. It seems to be the best system we can come up with. So what are the problems?

One of the challenges in the system of peer review is the long delays this process can incur. It can be difficult for journal editors to find suitable reviewers willing to take on the work. Not only do many academics these days find themselves confronting ever-increasing workloads in their official jobs, but in most disciplines they are asked to do this extra work for no pay and no recognition by their institution. Editors must rely on the ‘gift economy’ operating in academia, hoping that reviewers will subscribe to the belief that what goes around, comes around – by doing their share of reviewing, someone else will review their own article when they later submit to a journal. Further delays occur when well-meaning reviewers agree to do the work, and then find themselves overwhelmed by other tasks and responsibilities. From an editor’s point of view, very subtle nagging skills are needed to coax this voluntary work out of reviewers; from an author’s point of view, a great deal might be hanging on the outcome of the review.

And when those reviews do finally arrive, how helpful are they? In most areas, the standard practice is blind review – double (where the identities of both author and reviewer are anonymous) or single (where the identity of the reviewer is unknown to the author). In theory this anonymity is sensible, and protects the identities of reviewers so that they can be frank about their assessment of manuscripts without risking damage to their own careers. Unfortunately, this anonymity sometimes allows those reviewers to be vicious in ways that they might consider highly inappropriate if they were to speak openly to the authors.

Whether the reviews turn out to be positive or negative, they are really just two or three people’s points of view – a fourth reviewer may want something else again. It’s perfectly possible to get contradictory reports from different reviewers, suggesting that there is always an element of chance in what ends up getting published. Even with the best intentions of attempting to be objective and constructive, reviewers can submit entirely different reviews of the same piece of research – they may have particular interests, specialised knowledge, or be focused on different aspects of the writing.

There are moves afoot to try to solve at least some of the weaknesses of peer review. One response has been to implement processes of ‘open review’, that is, where the identity of the reviewer is made public and reviews themselves published. While this might encourage more courteous behaviour on the part of the reviewer, the potential risks associated with a junior researcher criticising someone with a big reputation in their field remains. In some disciplines, everyone has a pretty good idea of what projects are being undertaken by other research groups and where the funding went, so that author identity is a matter of informed guesswork if not overtly known; in these situations, open review dispenses with the pretence of author anonymity.

Post-publication review is another model that might be useful. This allows publication of research and then invites anyone who is interested in the topic to review the work. Such an approach fits well with contemporary practices of commenting on social media. While this system might draw some ill-considered reviews and may or may not be anonymous, on the whole it would seem to be a good way of encouraging debate and ongoing conversations in the field.

In an era when research output is endlessly measured and quantified, the work of reviewing that output could perhaps also be measured in order to provide reviewers with more reward for their effort. Publons is one organisation that is trying to make it possible for reviewers to get some credit for the work they put into reviewing; another is the ‘R-index’ suggested by Gero and Cantor. These are both ways of recognising the work of reviewing as having a measurable ‘impact’ and contribution to the development of the discipline and the dissemination of knowledge.

All these concerns are becoming ever more pressing as the move towards open access gains momentum. As the whole landscape of academic publishing is changing, these are important questions for all researchers to consider, and pose major challenges for doctoral candidates, their supervisors and learning advisors supporting them. What’s your experience of peer review? Do you have any solutions to the current problems, or ideas about how the system might be improved? (Which reminds me, I’d better get back to finishing the review that is waiting on my desk…!)

Scholarly editing and networking

Tags

co-editor, editing skills, networking, Peer review

by Cally Guerin

There’s lots of advice to doctoral students about how important conference attendance is for networking, but not everyone finds this easy. Personally, I’ve never been very good at bouncing up to strangers to introduce myself, or breaking into the tight huddle of buddies chatting during teatime at conferences, so can understand why many find this daunting. I also used to think that the concept of “networking” was a touch grubby – as if it described the unpleasant schmoozing of people who were being friendly just to see what they could get out of others. Then I realised it meant making an effort to get to know your community, which changed my attitudes completely.

As well as conference attendance, one of the most effective ways I’ve found to network and build longer-term collegial relationships is through editing – by working with others on collections of essays. It started when I was a postgrad and volunteered on the journal that was published out of my department at that time. I learnt a lot about what to look for as a subeditor or proofreader. Nick Hopwood’s (2010) article on Doctoral students as journal editors does a great job of articulating the value of non-formal learning afforded by this kind of academic work, and Pat Thomson et al. (2010) also develop related ideas in detail.

Over the years, I have also been involved in a number of book-length projects as a co-editor. Yes, it can be quite a bit of work; and yes, this work is rarely acknowledged by the formal university structures that measure output (such as the ERA in Australia). Editing anthologies or collections of academic papers is usually unpaid, relying on the “gift economy” that remains a significant part of academic life (see, for example, Antal & Richebé, 2009). Yet I continue to do this kind of academic writing work because it brings me other kinds of benefits that feed into the rest of my work that is recognised by the institution.

What I do gain from being involved in such projects is the opportunity to learn a lot about current research, closely reading papers that I otherwise might not come across. It’s also a great way to hone the skills of editing and of peer review. Noticing and articulating how papers can be strengthened forces the reader to think carefully about the research and the writing. Learning how to do this in way that keeps authors on board with the project (and without the “protection” of blind review) is quite different from standard journal reviewing or providing feedback on students’ writing as a supervisor. Through these projects I’ve also learnt much more about how the publishing industry works – how to put together a book proposal, how to market it, and how to target particular audiences.

But what I value most in all of this has been the opportunity to develop collaborative relationships with co-editors and contributing authors along the way. Working alongside others, doing something productive together, has given me a way of networking that builds ongoing relationships. The people involved in one project may well suggest ideas for the next; others will pass on information about events related to the topic of the book. Gradually, a community of like-minded academics forms and shares knowledge about the discipline.

Of course, there is much that can go wrong in undertaking tasks of editing or co-editing. There’s the risk of offending authors by editorial decisions; of letting others down by not meeting deadlines; of insurmountable differences of opinion about how things should be done. So far I’ve been lucky, and have perhaps also learnt along the way (or, more accurately, have been taught by my co-editors and authors) how to communicate clearly in order to avoid these sorts of problems.

Nevertheless, I’d encourage doctoral candidates to take up opportunities for volunteering in helping with editing projects, whether they are special issues of journals or edited books. There’s much to be gained from getting involved – a risk worth taking.

References

Antal, A.B., & Richebé, N. (2009). A passion for giving, a passion for sharing: understanding knowledge sharing as gift exchange in academia. Journal of Management Inquiry, 18(1), 78-95.

Hopwood, N. (2010). Doctoral students as journal editors: non‐formal learning through academic work. Higher Education Research & Development, 29(3), 319-331.

Thomson, P., Byrom, T., Robinson, C. & Russell, L. (2010). Learning about journal publication: the pedagogies of editing a “special issue”. In Aitchison, C., Kamler, B. & Lee, A. (Eds) Publishing pedagogies for the doctorate and beyond. Routledge.