Research Excellence Framework 2028
Response to consultation on initial decisions from the UK education funding bodies
In June 2023 the four funding bodies (Research England, the Scottish Funding Council, the Higher Education Funding Council for Wales and the Department for the Economy, Northern Ireland) published their decisions regarding the design of the 2028 Research Excellence Framework (REF) indicating areas in which consultation was sought from those in the HE sector. The full document can be found here: Research Excellence Framework 2028: Initial decisions and issues for further consultation (jisc.ac.uk)
As indicated in the document (para. 25), the guiding principles for the review and redesign of the REF were to ensure the robustness of assessment outcomes, address the impact of the REF on research culture at HEIs and to reduce, where possible, the bureaucratic burden of the REF exercise.
The Association for Art History submitted a response to the consultation which addressed issues which we considered most relevant to those in our field. For the response, the AAH sought input from art history departments of varying sizes and geographical locations. We also coordinated with colleagues at the Arts and Humanities Alliance (a group of some 50 subject associations and learned societies in the UK) regarding the response as well as with those from other humanities disciplines. Rather than replicating what is in the Arts and Humanities Alliance response or making broad responses on the REF, we chose to send a more focused response reflecting the concerns of art history departments and our own concerns.
Following are the decisions for which consultation was sought, some specific questions posed (both in Italics), as well as our answers.
1. Volume Measure
The initial decisions indicate that HEIs will not submit staff to REF 2028; instead, staff data will be gathered directly from HESA to determine the volume. REF 2021 staff eligibility criteria will be used to identify staff posts that contribute to the volume measure, and the funding bodies will calculate the volume using an average FTE of eligible staff across AYs 25/26 and 26/27.
What practical challenges may institutions face in implementing these changes? What would be the impact of these changes on individual researchers and particularly those with protected characteristics?
With the reliance on HESA data for the two academic years proposed and with the final volume measures not being announced until 2027, we are concerned that units will have a difficult time planning for and putting together the requisite number of impact case studies (ICS). This will have a greater affect on smaller units who may be near the cut off point for submitting one or two case studies, representing a strain on their limited resources with little time to organise a doubling of the necessary work to produce the case studies. The uncertainty, until 2027, of the volume measure and, as a result, the number of ICSs and outputs required will lead to constant monitoring of HESA data and may impact hiring decisions particularly where a new member of staff would mean that additional ICSs are required. One unintended outcome may be that the proposed volume measure may encourage units to hire or move staff into teaching only contracts (i.e. without a ‘significant responsibility for research’) in order to avoid the administrative burden of the additional ICSs. This may have a greater impact on those with protected characteristics and ECRS who may be getting their first jobs in an HEI.
2. Submission of outputs
REF 2028 will fully break the link between individual staff members and unit submissions.
Institutions will be able to submit one output to multiple submissions, provided it falls within the UOA descriptors. In REF 2028, HEIs will be required to submit 2.5 outputs per FTE of volume-contributing staff in each disciplinary area where they have activity. Institutions may submit outputs produced by any staff member where there is a demonstrable and substantive link to the submitting institution within the REF assessment period. Institutions will be able to submit one output to multiple submissions, provided it falls within the UOA descriptors. There will be no minimum output requirement for volume-contributing staff there is no need for a process to account for individual equality-related circumstances.
What impact would these changes have on institutions in preparing output submission? What impact would these changes have on individual researchers and particularly those with protected characteristics? What might be the unintended consequences of these changes?
The concept of ‘de-coupling’ outputs from individual staff members, while well-intended, raises concerns regarding the personal accountability of research (where, in the humanities, the vast majority of outputs are by single authors) and regarding research culture and development within units. While we welcome the principle of removing the requirement of one output per individual, we are concerned that the proposed approach would encourage a two-tiered system of staff whose outputs are identified as more valuable (i.e. chosen for submission) and others who will be left with a clear sense that their research and outputs are less valued in their units. This may retard the professional development and career prospects for the excluded staff who will not benefit from the time and potential departmental resourcing directed to those who are contributing to the submissions. And it may lead to or enhance a culture of inequity within units.
Regarding the impact on individual researchers, particularly those with protected characteristics, clarification will be needed on how the output of those on short-term contracts will be accounted for. If a researcher is on a contract for under a year, will their tangible outputs from the time of the contract only be eligible for a unit’s submission? And, if so, how will this be reconciled if the researcher was on another short-term contract prior to the contract in which their work was published, where most of the research would have been conducted and/or supported? Will this make those on precarious contracts less viable in the marketplace if the results of their research will not be submitted during the time of the short-term contract? If that is the case, this provision would serve to prejudice those with protected characteristics who make up a larger percentage of the “precariat” than do colleagues in full-time, permanent employment.
Regarding the submission of one output to more than one submission, we would welcome further explanation as to how the assessments of these submissions will be made, how these will contribute to unit of assessment scores and whether HESA data will help to determine where a unit makes these submissions and how much choice they will have in this determination. We would also like to know if cross referral from REF 2021 has been done away with or will exist in some form in REF 2028.
As to the proposal that the outputs of any staff member with a substantive link to the submitting institution, we anticipate both benefits and detriments to individual researchers. In our field, we very much welcome the inclusion of outputs of curators working at HEIs as their work contributes importantly to both knowledge and understanding and to engagement and impact. However, this approach may very well prove to be detrimental to those on teaching-only contracts or those (academics) in non-academic or ’research enabling’ posts (a term for which we would welcome a definition/examples) who, by circumstance of employment, will have less time to develop their research and therefore will be less likely to be included in REF submissions. While it would be unfair to require those in non-academic positions to submit to the REF as they are not on contracts which accommodate research, the proposal to make them eligible will create an expectation or an aspiration that they contribute to the submission to advance their careers without the infrastructure or support from their institutions to create outputs for inclusion. This could, in turn, lead to exploitative practices with an inordinate impact on those at early stages of their careers or those with protected characteristics. Perhaps a way to address this would be to require units to explain, possibly in the evidence statements, how/whether they have supported the research/outputs of non-academic staff who are included in submissions and how this is taken into consideration in the hiring practices of the unit.
Some of the concerns noted above, i.e. a fear of exploitative practices, the portability of research, identifying at an early stage who is more worthy of time/attention/funding, would apply equally to making PhD theses and post-graduate research eligible for inclusion in the submissions. We, therefore, have misgivings about this proposal. If it is incorporated into the final REF guidelines, we would like to see the definition of a demonstrable and substantive link to the submitting unit to be strengthened, for example by requiring a minimum one-year contract which would discourage institutions from issuing nine-month contracts which would suffice under the current proposal.
3. Engagement and Impact
The Impact element of the REF will be expanded to Engagement and Impact.
This element will include assessment of impact case studies and a structured explanatory statement which sets out the wider contribution research activities to society and the economy. The structured statement will make up at least 20% of the quality sub-profile for this assessment element. The funding bodies intend to weight the statement on a sliding scale, proportionate to the number of case studies submitted to ensure that its contribution to the sub-profile is no less than that of a single impact case study. The number of case studiesrequired in each submission will be determined by the average FTE of volume-contributing staff in the unit. Institutions will be required to submit a minimum of one impact case study per disciplinary submission. The funding bodies propose to revise the thresholds between case study requirements.
Our concerns in these areas are mainly around the burden put on smaller units and how their contributions are fairly measured against larger humanities departments and those in other disciplines.
We are encouraged to see the addition of the consideration of engagement in what was the impact assessment element as, among other things, we think it will enable a wider range of research to be featured. And we welcome further consultation on metrics, etc. for the explanatory statement. We have questions as to how the metrics chosen will adequately measure the contributions of humanities subjects such as art history to society. Would they be disadvantaged, for instance, when compared to some STEM subjects the impact of which may be more readily quantifiable?
We are concerned about the reduced weighting given to outputs vis-à-vis the statements as the former serve as an essential indicator of academic excellence, rigour and robustness and, as such should not be lessened within the assessment areas. We are also concerned that the addition of two statements will increase the administrative burden which will be felt, disproportionately, on smaller units.
A greater burden, in proportion to the proposed number of required case studies, will also be felt by smaller departments regardless of the discipline; however, the workload issue will be more acute for art history departments, many of which are on the border between having to submit one or two case studies. As an example, a unit that has ten staff (not an uncommon number in our field) will be required to submit two case statements, one per every five staff members. A unit at the top of the next category with, say 39 staff (not at all common for smaller humanities disciplines) will mean that they will submit one case study for every 13 members of staff, a much lesser burden on their resources. As smaller departments will struggle with the requirements, it may be more equitable to raise the threshold to produce two statements to 24.99; three to 44.99 and four to 64.99 which would serve to more evenly spread the workload across the various units. When considering that the statement on contribution will also favour larger departments over smaller ones, we ask for further reflection on how these requirements can be less burdensome, in proportion, for smaller units.
4. Units of Assessment
REF 2028 will retain the REF 2021 Unit of Assessment structure and will include advisory panels on Equality, Diversity and Inclusion, and Interdisciplinary research.
The AAH did not submit a response to this provision as there was support for retaining the UoA structure from those in our field.
5. Impact of the Covid-19 pandemic
The funding bodies intend to retain the statements on Covid impact that were used in REF 2021, and to require some consideration of how Covid impacts have been addressed in output selection as part of Codes of Practice.
We welcome the recognition that the pandemic did not affect all researchers equally and that some with certain characteristics were more disadvantaged than others. And we look forward to further consultation on how Covid impacts have been addressed in the output section of the Codes of Practice. However, initially, we would like to see that the acknowledgement of the significant disruption of the pandemic is reflected in the outputs (2.5 average per FTE) and ICS requirements which are unchanged from REF 2021. It is fitting that measures to be taken into account to assess the impact of the pandemic include those where the Covid had a disproportionate impact, ie. On those with protected characteristics, those with caring responsibilities and, in our own field in particular, on curators and others whose long-term exhibition projects which may be negatively disadvantaged in terms of impact case studies.