By our Academic Uncles, Glen O’Hara and Andrew Dilley
In our last blog we gave a few pieces of hopefully timely advice for those seeking an academic post in the run up to the REF2014 submission date, which fell last December. In this as our next blog, we offer a broader overview of the Research Excellence Framework (REF) for those seeking academic jobs in the UK over the next few years. The REF plays a major role in shaping patterns of research and the recruitment of researcher-active staff in the UK, and a working knowledge of the REF is therefore an asset when seeking an academic job.
What is the REF?
For several decades public funding for research in the United Kingdom has been distributed through two mechanisms. First, grants for individual programmes of research are awarded by bodies such as the AHRC (Arts and Humanities Research Council) or the ESRC (Economics and Social Science Research Council). Secondly, ‘Quality Related’ (QR) funding is distributed by the Higher Education Funding Council for England (HEFCE), the Scottish Research Council, the Higher Education Funding Council for Wales, and the Department for Employment and Learning in Northern Ireland.
QR funding is allocated on the basis of the Research Excellent Framework (formally Research Assessment Exercise or RAE). HEFCE runs the REF on behalf of all UK funding councils—so what follows applies across the UK, at least for now. Disciplines (or ‘units of assessment’) are judged on the quality of their research, along with several other elements (‘impact’, research environment and the like), during defined assessment periods. These results then determine the funding allocation in the next period: the REF is intended to reward and sustain established pools of research excellence, while providing a stable ‘baseline’ of funding that is not subject to the chance and variation involved in constant bidding for individual awards.
The last assessment period ran from 2008 to the end of 2013, and this present one began on 1 Jan 2014 and will probably end at some point in 2019 or 2020. At the end of each assessment period, units submit four publications (or ‘outputs’) published within the period produced by each full-time researcher. Each institution can choose whether or not to include a particular researcher in their submission. Four expert panels are responsible for grading the submissions and each panel has slightly different rules. Each panel is, in turn, divided into subject-based sub-panels—History, taken with History of Art, has its own sub-panel. Each output is assessed by a member of the sub-panels of graded on a scale of 1*-4* (where 1* is ‘nationally recognised’ and 4* is ‘world leading’). The individual grades are anonymised. They are then combined with the results for impact and research environment (also graded by the sub-panels) to produce a ‘quality profile’: a score intended to reflect the overall calibre of the discipline or department within each university.
Why does the REF matter to me?
It matters for two reasons. First, it generates a sizable and relatively stable source of funding. The quality profile is used to distribute QR funding, and most historical research is currently supported by QR funding. Only research judged to be of 3* or 4* quality attracts funds and this is true across the UK. Second, the REF results generate status within and for universities keen to boast of their panoply of top-rated units.
Many universities are very keen to play the system to maximise both status and income. They put in place complex strategies to maximise their results, in particular to ensure they only submit staff whose research will generate a decent proportion of 3*s or 4*s. Many of the most important ramifications for Early Career Researchers are the result of this selectivity.
Quite simply, if you are employed in a UK university to conduct research, you will be expected to deliver a ‘REF Hand’. That means not just four publications between 1 January 2014 and 2019/20 (and possibly/preferably well before), but also four publications that the institution wishes to submit. Every selection panel will have at least one member who understands the REF (or thinks they do) and your research plans and job application have to show how you will meet REF criteria.
Listen carefully 007: Some important details and their implications
There are a few details that it is worth being aware of:
- Are all publications equal?
In principle all publications (that means anything in the public domain, including on a website) are judged equally. In practice many institutions (and a not inconsiderable number of historians) are very wary about non-traditional publications – particularly those that aren’t peer reviewed. Bear this in mind when planning research and constructing a CV.
- Do books count for two?
No. Some ‘outputs’ can count double in the REF if they take more ‘effort’ to produce than a standard output (for which, read: a journal article). For History and other related disciplines on Panel D (Arts and Humanities), a book submitted with a case for double weighting could be submitted with an ‘insurance piece’ in case the case for double weighting was rejected. So a book can count for two (and History sub-panels, appreciating the value of the book for the subject, have been supporters of double weighting), but it is not guaranteed. Equally, an apparently much shorter output (a database, for instance) could count for two if it took considerable effort to produce. All of these things are judged on a case by case basis after submission.
- What is overlap and how is it treated?
Overlap occurs where one output includes elements of another output. Classically in the case of someone early in their career, this might include an article that later appears (perhaps in adapted form) in a book. The rules of the last REF stated that where overlap occurred, the larger item would be assessed disregarding materials in the smaller items (so you only get credit for the same work once). Many institutions, however, resisted submitting any overlapping items at all, considering the rules ambiguous, or finding it hard to predict the grades that overlapping items would receive, leading to the oft-heard question: ‘how good is Blogg’s book without Chapter Five?’.
- What about Open Access?
There is a tremendous policy drive now to make publications (especially journal articles) freely available to the public, who, after all, funded much of the research work involved in the first place. This can be achieved either by making them available on the web at the point of publication (so-called ‘gold’ open access, often with the researcher or their institution paying a charge to publishers) or with a delay through an institutional repository (known as ‘green’ open access). This area is too complex to enter into here in detail (stay tuned for an upcoming post by History Lab Plus Chairs Kimm Currran and Cath Feeley), though a great deal has recently become clearer about how this will work in practice. The REF will probably require all journal articles and conference publications to be made available through an institutional repository. Open access requirements will not apply to monographs or book chapters. The ‘green’ route (where a version of the article is made available with a delay) is permitted without further payment to the journal as copyright holder. Certain crucial details have not been finalised, especially the length of the delay, or embargo, permitted under the green route. It will not be less than 12 months, and probably no longer than 24 months. The best advice at this stage is for those seeking jobs to keep an eye out for the announcement of the finalised rules and check that any journals you submit to comply with the ‘green’ open access rules. Non-UK journals may need to be prompted or asked to reveal their own policies and attitudes. Needless to say, you won’t be alone in finding this a puzzle: a lot of scholars are delaying submitting articles to journals until this is clear.
- Aren’t there dispensations for Early Careers Researchers?
Yes, but they may not help you now. Basically the REF considers your research career to start when you get an academic job with research in your contract. For the first (approximately) four years of your career you are permitted to make a smaller submission based on the number of years you have been employed to conduct research at the point of submission. So, if on that date you are one or two years in, you submit one output, three years in, two outputs, four years in, three outputs. So only those who get academic jobs after August 2015 will get any relief from the REF in 2019/20. As an aside, there are other dispensations. In particular, part-time staff are only required to make a reduced submission, and those going on maternity leave are allowed one fewer outputs. If you have special circumstances it would be worth checking how the REF treats them (see the links below).
- Do I have to have Impact (whatever that is)?
No. ‘Impact’ is assessed in the REF, and seeks to gauge the benefits generated by research findings in a unit of assessment for non-academic audiences (although some ‘impacts’ in universities also count). Each unit submits a certain number of case studies (two for the first 15 full time staff members and a further one for each 10 thereafter). Impact must be based on research published by researchers in the unit in the 20 years preceding the census date (1999 for the next REF). What this means is that not every researcher has to have impact – only a selection of those returned by the institution. The assessment of impact is new, and how this will work in practice is very unclear. It is more likely that the potential future impact of work as yet unpublished will be of interest to prospective employers (since this would count under the byzantine rules). If you might publish work once they give you a job that may go on to benefit academic audiences, that’s great. If you are not about to inspire a TV documentary, revamp a museum, or advise the UN, don’t worry too much. Emphasise the quality of your publications, but just also make sure that you think about how your work might benefit non-academic audiences in case you are asked about this in any interview.
So what can I do to maximise my chances of a job?
The first step is to realise that when applying for jobs, or at least permanent jobs, you will be judged by a panel which will be looking for four decent publications by the end of 2019 (and preferably a lot earlier), unless you are likely to be an Early Career Researcher—i.e. you get your first research job after August 2015. You need to plan your possible publications and make sure that they are clear on your C.V. The classic history ‘REF Hand’ is a book plus three articles/book chapters. If you have unconventional but substantial outputs (content-rich websites or databases, for instance) make sure that these are clearly explained on your CV and in your application. Make sure your outputs don’t overlap too much. Finally, be ambitious with your publications. Aim for good journals, substantial sounding book chapters, and decent monograph publishers. Don’t waste time with too many conference proceedings. Avoid large publications that are not peer reviewed. All of this will reassure a selection panel of that your work is good. Only publications after 1 Jan 2014 will count to the next REF. A track record will help, but clear future plans are essential. Keep an eye on the evolving rules on Open Access for journals, and make sure you tap reliable sources (subject associations, HEFCE’s website, etc) and not the blogosphere. There is a lot of misinformation out there.
Bear in mind that everything we have written is based on the rules for REF2014. They may change but until they do, selection panels will work on the same assumptions. Again, keep an eye out for any changes.
Before getting too disheartened, bear in mind that much of this would have been necessary anyway. A clear ambitious research agenda is the key to getting an academic job in any case. But more than anything, the REF makes it necessary to tie this to a clear set of planned publications.
I still think this all sounds byzantine, why doesn’t anyone try to change it?
Actually, they do. Organisations such as the Royal Historical Society, the British Academy, HistoryUK, indeed History Lab Plus, as well as individual universities and academics, try to shape the rules to make them more helpful – both to historians, and humanities scholars more broadly. This does sometimes make them even more complex, as officials try to juggle different demands, but also means that the rules on (for example) the dispensations for ECRs, maternity, and the like have been more generous than in RAE2008.
That said, those seeking academic jobs tend to be at the sharp end of the REF. It is far easier for someone with a permanent job to defy its requirements than for those without that elusive permanent contract, though threatening noises are often heard, in many different types of institution, about the consequences of non-submission even for very senior members of staff. But colleagues further on in their career arcs can usually point to a track record of prior success. Rightly or wrongly, it is hard to launch a career without convincing a prospective employer that you will ‘deliver’ for the REF. Forewarned is forearmed!
Further Information: The rules of the last REF are available in two documents, Assessment framework and guidance on submissions (REF 02.2011) and Panel criteria and working methods (REF 01.2012). The latest from HEFCE on Open Access can be found here [http://www.hefce.ac.uk/pubs/year/2013/201316/]. The rules of REF2020 may well change.