Papers
List
of papers
Responses to Questions For Communication, Cultural and
Media Studies (UOA 65)
Question 1
How far do you agree that the descriptor provided
by the sub-panel describes the main subject areas of the
UOA?
(Note: following a consultation conducted by the
HE funding bodies, the units of assessment for the 2008
RAE are fixed and are not subject to change – see RAE
02/2004 Panel configuration and recruitment and RAE
03/2004 Units of assessment and recruitment of panel members.
Respondents should focus on whether the sub-panel’s
description of the UOA and of its boundaries with other
UOAs is comprehensive.)
Strongly agree / Agree / Disagree / Strongly
disagree / Don't know
Comments:
In general, MeCCSA welcomes the clearer and wider definition
of research now adopted and the greater standardization of
procedures and criteria compared to the last RAE.
MeCCSA is pleased to note the range of research forms and
outputs to be accepted by the sub-panel and assessed against
the indicators of excellence and degrees of quality described
in paragraphs 15 - 22 of the main panel statement.
Top of page
Question 2
How far do you agree that the sub-panel’s
proposed weightings for research outputs, research environment
and research esteem are appropriate to the UOA?
(Note: in line with RAE team guidance, research
outputs must be weighted at not less than 50% and esteem
and environment must each be weighted at not less than
5% and the three must sum to 100%; main panels are encouraged
to adopt consistent weightings for their sub-panels unless
variation is justifiable.)
Strongly agree / Agree / Disagree / Strongly
disagree / Don't know
Comments:
The weighting between components has left the Research Environment
at no more than 20%. This has raised concerns regarding the
impact of early-career researchers. The RAE has declared a
wish to minimize the benefits of tactical buying-in of big
names, and to reward those institutions which work hard to
bring on young researchers. However, at present, the only acknowledgement
given to this is in the provision that early career researchers
may not return four outputs. We would argue for the introduction
of a final, ‘balancing’ consideration. Where a
returning Unit is graded very high in its research environment
(that is, in all those processes whereby it supports and develops
those within its remit), but is returning over 50% of early
career researchers, in the final stage of judging quality,
Sub Panels should be invited to consider adding a weighting
for emergent potential in awarding its final profile.
In addition MeCCSA would like more explanation of how the
proposed 70% quality profile is to be broken down. How will
the 5% bands be related to the overall starring system? Will
the 70% itself be a conversion from a scaling system of 0-100?
Top of page
Question 3
How far do you agree that the sub-panel’s
range of indicators for excellence is appropriate for the
UOA in assessing the submitted research outputs?
Strongly agree / Agree / Disagree / Strongly
disagree / Don't know
Comments:
We would welcome a more detailed statement about how research
outputs will be allocated to the star grades. This would
be helpful in determining which items to select in each individual’s
submission and also in deciding which researchers to include
in the submission. For example:
-
Will the evidence provided in the ‘esteem indicators’ be
used to cross reference to the research outputs in order
to assess the degree of recognition that a work has enjoyed,
and therefore its quality? If so, how will this affect
the evaluation of very recently published work whose
impact is not yet measurable?
-
How will books that are designed to be used as student
textbooks be regarded? In relation to this point we would
want to draw the panel’s attention to a recent
Publishers Association conference ‘Publishing for
the RAE 2008 and beyond’ (University College London
- Wednesday 29th June 2005) where this issue was discussed.
John B Thompson, the Cambridge media sociologist, identified
academic publishers increasing reluctance to (or even
termination of) publishing scholarly monographs (because
the market is too small) and keen interest in ‘textbooks’ to
be at direct odds with the RAEs valuing of monographs
and devaluing of ‘textbooks’. Thomson argued
that the RAE had effectively stopped the writing of first
rate, research informed, textbooks by the best UK academics.
However, this was an area of publishing in which there
had, in the past, been some very important historical
achievements, especially in areas such as sociology,
literary studies, and history and which had represented
and articulated the relation of research and teaching
that should be at the heart of Universities. We would
argue that just because a book might be addressed to
students it should not obscure the degree to which they may
be works of research and original thought, particularly
within new areas of enquiry. Cultural and Media Studies
has an historical link, in its founding texts and arguments
(Williams, Hoggart, Hall, Whanell, Thompson), with educational
processes and a popular but non-reductive address to
readerships. The panel for UOA 65 might wish to take
a more explicitly inclusive view of the research content
of ‘textbooks’ in the manner that the UOA
62 (History) panel has indicated. We quote from the History
panel’s draft guidelines: ‘1. No form
of output will be regarded as intrinsically inferior
to any other. 2. Where there is a visible contribution
to research, the following forms of scholarly outputs
(listed alphabetically) will be evaluated: books,
including textbooks which incorporate considerable personal
research or substantially advance the subject area’
MeCCSA would also welcome detailed guidance on joint authored
pieces. We note that History UoA (62) paragraph 19.p.4 states 'joint
authored pieces will be treated as a single-authored piece…joint
authored output may be listed by more than one individual
in a department's submission'.
There is some concern at the way in which the RAE will deal
with the question of über-texts, that is, submitted outputs
which claim to be worth more than one. We understand the motivation
behind this – that the RAE does not wish to disadvantage
or discourage long-term and ambitious research plans. However,
the way in which this is currently being proposed, will turn
it into a high-risk strategy. As currently proposed, it will
be in the hands of a returning University to claim that one
output should count for two or more, but if their claim is
not accepted, that person will be counted as having returned
less than four, and thus penalized and awarded 0* for the
missing item(s).
We would argue for a change to this. If the
RAE wishes to sustain this principle, then it should be
amended to remove this risk, so that an institution could
submit four items, but make a claim that one of them should
be considered for more than one. Should the claim be rejected
by the Sub-Panel, the remaining items would still be available
for consideration. Should it be accepted, one or more of
the others could have been marked as the one(s) to be deleted
from consideration.
MeCCSA notes that departments are not required to include
all their eligible staff in their submission but there is
not necessarily a financial incentive to exclude people.
We would like this to be made more explicit and explained
in more detail in the sub-panel statement.
More information is sought about how Category B staff will
figure in the assessment of outputs and the scoring of profiles.
The sub-panel 65 criteria refer to scrutiny of Category A and
C staff outputs but there does not seem to be reference to
Category B. This could affect departments where some staff
who will have retired by the census date will be entered.
Top of page
Question 4
How far do you agree that the sub-panel’s
range of indicators for excellence is appropriate for the
UOA in assessing the research environment component of
a submission?
Strongly agree / Agree / Disagree / Strongly
disagree / Don't know
In general, we would like clearer indications of how this
is to be reported. There is a generally box-ticking approach
taken here and we have no indication how the absence of information
on particular points will be treated, should that information
not be deemed to be relevant for a given department’s
submission by those making the submission. We question the
relevance of supplying information under:
14. Strategy c. iii Research grant applications etc, where
it is asked for the numbers of grant application and the
number of successful applications. We cannot see
how asking for information on unsuccessful applications
can do anything other than damage a department’s
standing.
Paragraph 17 notes that ‘The assessment will be focused
on outcome rather than income’. The paragraph then
goes on to emphasise the importance of research income, and
in particular, it says the sub-panel will ‘take account
of the total number of grants relative to the size of the
department’. We question the relevance of this crude
numerical criterion, which presupposes that research income
is necessarily relevant to a quality assessment.
Furthermore, criteria that equate getting money from research
councils with high level research are problematic. The end
result is that certain kinds of research are rewarded twice
further increasing funding differentials.
Strategy f.i. Other research activities, The Achievement
of research staff etc. This would seem to be a redundant
category as information will have been supplied on this elsewhere
in the submission.
We ask that Paragraph 14(d)iv is redrafted to include NGOs
and that Paragraphs 14(f) iv & v are redrafted to include
organisations of civil society.
MeCCSA welcomes the undertaking in Paragraph 13 that the
numbers of research-active staff submitted and their levels
of experience will be taken account of. This may discourage
the exclusion of individuals from departmental submissions
which is potentially damaging for individual careers. Further
advice on how this will be factored-in would offer more assurances
and lead to less ambiguity and second-guessing.
Paragraph 15(c) notes that the sub-panel would 'allocate
a further 25% on an holistic basis'. This is rather vague.
MeCCSA would welcome some examples of indicators that will
be employed.
Top of page
Question 5
How far do you agree that the sub-panel’s
range of indicators for excellence is appropriate for the
UOA in assessing evidence of esteem within a submission?
Strongly agree / Agree / Disagree / Strongly
disagree / Don't know
Comments:
Paragraph 17 notes that ‘The assessment will be focused
on outcome rather than income’. The paragraph then
goes on to emphasise the importance of research income, and
in particular, it says the sub-panel will ‘take account
of the total number of grants relative to the size of the
department’. We question the relevance of this crude
numerical criterion, which presupposes that research income
is necessarily relevant to a quality assessment.
Paragraph 19c Benefit. We wish to point out that while this
paragraph says ‘examples may include’, all those
included are in fact valued in money terms. This encourages
the impression that the criteria to be applied will be narrow.
Paragraph 19(b) Influence. We would like this to include
executive and non-executive positions on boards of relevant
organisations of civil society.
Top of page
Question 6
How far do you agree that the sub-panel has identified
appropriate criteria for assessing the vitality and sustainability
of the research described in each submission, including
its criteria for assessing the contribution of researchers
at different career stages?
Strongly agree / Agree / Disagree / Strongly
disagree / Don't know
Comments:
We would like to repeat the comments made in section 2 above.
The weighting between components has left the Research Environment
at no more than 20%. This has raised concerns regarding the
impact of early-career researchers. The RAE has declared
a wish to minimize the benefits of tactical buying-in of
big names, and to reward thoseInstitutions which work hard
to bring on young researchers. However, at present, the only
acknowledgement given to this is in the provision that early
career researchers may not return four outputs. We would
argue for the introduction of a final, ‘balancing’ consideration.
Where a returning Unit is graded very high in its research
environment (that is, in all those processes whereby it supports
and develops those within its remit), but is returning over
50% of early career researchers, in the final stage of judging
quality, the Sub Panel should consider adding a weighting
for emergent potential in awarding its final profile. Without
provisions such as the above the RAE will continue to have
a serious impact on the academic culture of young scholars.
Top of page
Question 7
If relevant in this UOA, how far do you agree that
the sub-panel’s criteria and working methods are
appropriate for identifying and assessing applied research
equitably alongside other forms of research?
(Note paragraph
37b of RAE team’s Guidance to panels)
Strongly agree / Agree / Disagree / Strongly
disagree / Don't know
Comments:
In Paragraph 23. MeCCSA would like to include recognition
of policy submissions to government departments, public bodies
and NGOs as valid examples of applied research.
Top of page
Question 8
If relevant in this UOA, how far do you agree that
the sub-panel’s criteria and working methods are
appropriate for identifying and assessing practice-based
or practice-led research equitably alongside other forms
of research?
(Note paragraph
37b of RAE team’s Guidance to panels)
Strongly agree / Agree / Disagree / Strongly
disagree / Don't know
Comments:
Practice as research has been covered by the generic heading
of Applied Research and the same text appears in each sub-panel's
document. We are concerned that the specificity of arts practice
is not well represented and that calling everything an 'artefact'
is too loose. Further clarity is required.
Top of page
Question 9
General comments on any other aspect of the sub-panel’s
criteria and working methods. Where appropriate, respondents
might wish to comment here, for example, on any non-standard
data or data analyses that the sub-panel has requested:
Working methods
Paragraph 33b. We would like further clarification as to
how research outputs will be selected and for reassurances
that the process followed will not be to the potential detriment
of researchers. Further clarification of how members of the
panel assigned to the task of examining particular outputs
will divide their labour would be of help. Will they read
the same outputs or different ones?
MeCCSA notes that care has been taken over questions of
conflict of interest. However, while this will work effectively
at the point where assessments of individual institutions
are taking place, it cannot work so effectively at the point
when Sub-Panels, and indeed Panels, are discussing the relations
between assessments – otherwise,
at this point, meetings will be inevitably be stripped of most
members. MeCCSA asks that the panel/sub-panels clarify how
they plan to manage this final stage of the assessment process.
MeCCSA
is also concerned about confidentiality. Although the star-rating
of individuals is intended to be confidential it is difficult
to foresee how this is possible in a field as small as media,
communication and cultural studies. We would like further
information on the processes of ensuring confidentiality
of individual staff ratings.
Clarity is needed too concerning
the role of the wider sub-panel in debating the assessments
of the individual teams – where, for instance, sub-panel members might
disagree with the assessment of particular outputs by those
allocated to assess them.
General Comments
It is crucial that departments have the earliest indications
possible of the length of RA5 and how this relates to vast
array of information required. The likely effort of preparing
RA5 is quite disproportionate to the final weight assigned
to it in the overall quality assessment.
We also wish to underline the widely felt concern in the
field that the RAE process is going to lead to a further
concentration of research capacity, which we do not regard
as the best way to ensure pluralistic and creative advances.
The RAE has had a disastrous impact on the UK higher education
system, leading to the closure of departments with strong
research profiles and healthy student recruitment. The RAE
has been responsible for job losses, discriminatory practices,
widespread demoralization of staff, the narrowing of research
opportunities through the over-concentration of funding and
the undermining of the relationship between teaching and
research. This is of particular importance for CCMS, given
its still relatively early stage of development and the need
to maintain as wide a research base as possible for the nurturing
and growth of research talent.
Despite attempts to discourage games-playing the current
exercise appears to be stimulating even more competitive
recruitment and 'restructuring' driven purely by attempts,
ill-fated or otherwise, to maximize RAE income. As in the
past, the exercise will, without doubt, have a serious impact
on departmental and institutional practices, research planning
and collegiality, further distorting and disrupting the system
and devaluing the professional contribution of many staff
to teaching and research.
Finally, although MeCCSA appreciate that the decision on
funding is entirely out-with the purview of the sub-panel
we would like to register our continued frustration that
departments will have no idea in advance of how assessment
scores will translate into financial awards.
Top of page
|
