Papers
List
of papers
Response to Joint Funding Bodies’ Review of Research
Assessment
November 2002
This Association represents teachers, researchers, and students
in its fields within UK Higher Education. Its predecessor
body was instrumental in arguing for the formation of a panel
within the Research Assessment Exercise that could properly
recognise and assess research in our fields. This panel (65)
was created for the first time in the 1996 Exercise. Previously
the field had been inadequately assessed as a division of
work within a broader category of research including information
science and other, rather disparate, research areas.
Our response to the Research Assessment Exercise is guided
by the following principles:
-
Any such exercise must be designed to secure improvement
in the quality of research in universities. The costs,
both human and material, of undertaking the Assessment,
for both units of assessment and for those acting as
assessors, must be less than the demonstrable benefits.
If the primary purpose of the RAE is for resource allocation,
then it has already gone further in concentrating resources
than we believe to be healthy or productive, and in our
judgement, further assessment leading to further concentration
could only damage rather than enhance overall research
quality.
-
The purpose of the exercise is brought into disrepute
when it does not fulfil the declared purpose of supporting
excellence in research. The 2001 exercise clearly demonstrated
improvements in the quality of research, with more work
being judged to be at national and international level.
In many instances, financial restrictions have meant
that this improvement was not rewarded, and this must
call into question the ability of staff to maintain and
improve current levels.
-
Many fields, ours included, do not always, or even predominantly,
require concentrations of either facilities or of people
to enable high quality research. Lone scholars, or small
groups of researchers, frequently make significant contributions
to research. Any Assessment Exercise must recognise this,
while the funding consequences of assessment must ensure
such activities are protected.
-
Our field is a relatively new and certainly dynamic
one. Assessment must offer maximum opportunity for the
demonstration of research excellence in newer departments
or groups, and must also reflect the frequent changes
in the location and dispersion of research activity across
the sector.
-
We regard good teaching and opportunities for research
to be inextricable. The QAA assessment of our field made
extremely clear the benefits of teaching programmes delivered
by staff with active research programmes. We would oppose
any consequences of research assessment which led to
damaging and unnecessary concentration of research, or
of a drift to a division between research active and
inactive centres.
-
Our field, like several others, incorporates a wide
range of approaches to “original investigation
undertaken in order to gain knowledge and understanding”.
We accept and endorse that definition of research. However
we feel further discussion, across a number of subject
areas, is required to move to a more widely accepted
and understood interpretation of this definition as it
applies to a diversity of work, including professional
practice and non-traditional forms of output. The relationship
between practice and research remains a problem to which
different panels seemed to take different approaches.
Further work needs to be done if a better understanding
is to be reached about how departments and institutions
which encourage a diversity of work are to be appropriately
assessed either in the RAE or outside it.
With these principles in mind, the Association offers the
following comments on the Review paper.
-
Of the four models for assessment offered we feel none
alone adequately captures any acceptable definition of
research quality. We do regard peer judgment as a sound
basis for assessment. This depends on the methods employed
being transparent and widely endorsed, and the experts
exercising the judgements being both accepted as such
by the constituency and adequately representative of
the diversity of work within the field. Such judgements
anchor the assessment in academic evaluation, a well
established principle of work within the academy, and
the least worst of the approaches suggested.
-
We do not, however, regard peer judgement as sufficient.
It will need to be buttressed by substantial elements
of evidential support for statements about the context
and culture within which research is conducted. For such
evidence we would reject many metrics. For example, citation
indices are a wholly inadequate, and largely misleading,
indicator of research excellence in our field. Input
factors such as external earnings can be an indirect
indicator of peer judgement, but are less significant
by far than the quality of outputs.
-
We cannot envisage how a system based on self assessment
could be effective. Either it would depend on the extensive
use of auditable metrics, which we regard as wholly insufficient,
or it would be based on the judicious use of rhetoric
and promotion, which would simply require a further assessment
exercise to adjudicate.
-
It would be beneficial and equitable if assessment covered
all funding council supported staff who are required
to undertake research. To facilitate this strategy and
to avoid penalising staff unfairly, careful consideration
needs to be given to the criteria for defining both the
nature of research and of the submissions expected. In
this area, two specific problems need to be carefully
addressed. The first relates to practice work as outlined
in the principles above. The second problem relates to
staff who during the period in question have been developing
work rather than bringing it to fruition. At the moment
they may either be pressured to produce work prematurely
or not be entered. In this context, it may be helpful
to strengthen those elements of the assessment which
relate to the culture of research and the support for
specific long term projects, however they are funded.
-
The distribution of resources between subject areas
cannot be derived from the RAE alone or even primarily.
We have previously submitted to HEFCE evidence to suggest
under-funding of our field because of the insufficient
allocation to what is termed our ‘subject pot’.
This was corrected in relation to teaching following
work by the Media Studies Advisory Group, but has not
been adjusted for research. The allocation across subject
areas should be on the basis of the demonstrable average
costs of research in given areas, which, while difficult
to establish, is not impossible, and by the volume of
research active staff.
Top of page
|
