The Research Excellence Framework (REF)

research.jpgUnless you’ve been asleep in the library for the past few years, you’ll almost certainly have heard about the REF (Research Excellence Framework); but you might be less aware of exactly what it is, how it affects you as a researcher, and what you need to do to prepare. In this post I’ll highlight the key points about the REF and indicate some sources of further information.



The REF is the new system for assessing the quality of research in Higher Education institutions in the UK (previously RAE). It’s used to determine funding distribution to universities from the Higher Education Funding Council for England (HEFCE). It’ll be completed in 2014, but the deadline for UK HE Institutions to make submissions is 29 November 2013. The assessment evaluates: research outputs (65%); impact (20%); and research environment (15%).

It is “research outputs” that are most relevant to PhD and ECRs’ career planning, job applications, and publication decisions, but it is worth being aware of Impact requirements, which I have included further on in this post.



man_with_question_mark-blueThis depends on where you’ll be employed on 31 October 2013. If you’re a PhD researcher and will still be a PhD researcher in November 2013: no. If you’re in a teaching-only position, e.g. teaching fellow, casual teaching contract: no. If you’re academic staff at a UK higher-education institution, you will be eligible if: you have a contract of 0.2 FTE or greater in a submitting HE institution on the census date and your primary job description is ‘research only’ or ‘teaching and research”. Research assistants (e.g. co-investigator) are not eligible to be returned to the REF. The exception is if you are named as principal investigator or equivalent on a research grant or significant piece of research work, and satisfy the other criteria for academic staff. Visiting professors, fellows and lecturers employed by other HEIs are not eligible.

It is usually clear on job descriptions if the post is teaching-only, research-only, or both, and it is typically stated if the post-holder will be retuned for the REF. If you’re applying for such jobs, it’s important to know what will be expected of your REF submission as this will have implications for the hiring process, and therefore for your career planning now.



ChecklistEach member of staff included in the REF submission is required to put forward a number of “research outputs”. Research outputs are the products of ‘a process of investigation leading to new insights’ that have been brought into the public domain between 1 January 2008 to 31 December 2013; this is typically books and journal articles, and depending on your field might also include conference contributions, physical artifacts, exhibitions and performances, other documents and digital artifacts (including web content). Your PhD thesis or any work submitted for a degree is not eligible, but a monograph or other publications based on your PhD would be eligible if it fulfilled the above criteria.

For most staff submitted to the REF, 4 outputs are required; however, this is lowered for early career researchers (and in some other instances, e.g. career breaks and maternity leave). Early-career researchers are those who fulfill the usual requirements for eligibility, but started a career as “independent researchers” on or after 1 August 2009; “Independent researcher” is the point at which you held a contract as described above, or acted as PI/equivalent on a significant research project. If you have been on teaching-only contracts or other non-eligible contracts since finishing your PhD but have continued research “on the side” (i.e. unpaid), then you do not count as an independent researcher until you enter a post as above.

The ECR discount depends on the number of years that you have been an early-career researcher:

Date at which the individual first met the REF definition of an early career researcher: Number of outputs may be reduced by up to:
On or before 31 July 2009 0
Between 1 August 2009 and 31 July 2010 inclusive 1
Between 1 August 2010 and 31 July 2011 inclusive 2
On or after 1 August 2011 3


Submissions will be assessed using a process of ‘expert review’ to award an overall quality profile to each submission, based on a star rating system: 4* is the highest award, for research that is “world-leading in terms of originality, significance and rigour”; the lowest rating is 1* for research that is “recognised nationally in terms of originality, significance and rigour” (see here for full information).



Sprout_LightbulbNadine has covered public engagement and impact in the previous post, but there are some specific requirements to be aware of in the context of the REF as not all public engagement counts as impact.

The REF guidelines define impact as: ‘an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia’ (“Assessment Framework”, p.26).  Crucially, impact must be underpinned by high-quality original research, and it must be measurable. Impact is assessed for its “reach and significance”, and scored on a 4* rating system (see here for full information).

Whereas research outputs are assessed for every eligible researcher, Impact is assessed via a selection of case studies that submitting departments put forward: this means that not every researcher has to have an impact profile in the way that they must have a publication profile. This doesn’t mean that you can decide to ignore impact, however! It is expected that the weighting given to impact will increase in future REF cycles; although an ECR wouldn’t be expected to have a fully-formed impact case study, you will be much more attractive to employers if you can demonstrate a developing profile of public engagement that has the potential for future impact.



TeamworkIf you’re currently applying for academic positions that will be REF-returnable then be aware that a hiring committee will prioritise a high-quality publication profile that can make a strong contribution to the REF submission.

If you will be eligible for a discount as an ECR then a hiring committee will factor this into their analysis of your profile, but quality and quantity both count: one very strong submission would be better than a number of lower-ranking items, but the optimum is to have a number of good quality pieces to choose from. You should demonstrate awareness of the REF and surrounding issues in your application form and at interview, and applications should clearly state the number of research outputs you will have ready for the REF (including secured contracts that will be published before the deadline). See advice from here:

If you don’t have a REF-returnable profile now, then it’s almost certainly too late to change this situation given the length of time taken to publish work. That’s not to say that you shouldn’t apply for jobs: there are a wide range of academic jobs available and not every employer will be prioritising REF return in their hiring process, especially if you go for teaching-only posts. Furthermore, now is the time to start thinking ahead to the next REF cycle.



cface338Universities are already starting to think beyond the REF 2014 to the next assessment exercise which will probably be carried out in 2019. Anything published after the 2013 cut-off point will be eligible for the next REF, so you can start planning a strong profile of work that will take you forward into the next submission cycle – and if you know you won’t be submitted for the REF 2014, then it might be worth holding off from publishing that 4* piece of work until after November 2013! Target where you publish: consider journal quality, scope and impact factor (see Nadine’s post on publishing and Warwick’s library series on publishing in journals). Some panels will make use of citation data in their assessment in 2014 and this is expected to increase in the next REF, particularly for STEM subjects. If you’re in a subject that uses citation data then be aware of how citation data works when choosing a publisher. Start building up a portfolio that demonstrates awareness and eagerness to participate in public engagement activities, as this will put you in a good position for future impact ahead of the next REF. For full details of everything I’ve talked about here, HEFCE’s guidelines can be downloaded here and please feel free to ask any further questions in the comments below!