Referring concerns to reviews
Introduction
1
This guide outlines how the Quality Assurance Agency for Higher Education (QAA)
can refer concerns identified through its Concerns Scheme to review teams for
consideration.
Background
2
The Concerns Scheme allows QAA to investigate concerns about the standards
and quality of higher education provision, and the information that higher education providers
produce about their learning opportunities. Where there is evidence of weaknesses that go
beyond an isolated occurrence, and where the evidence suggests broader failings in the
management of quality and standards, we can investigate. These concerns may be raised
by students, staff, organisations, or anyone else in a formal submission to QAA.
3
The Concerns Scheme has four phases:

Screening: to determine if the issues raised in the submission meet the remit of
the Scheme
Initial inquiry: to test the submission, gather further information, and invite a
response from the provider
Full investigation: to examine the evidence and determine if it reflects
serious systemic or procedural shortcomings, resulting in a published report
and recommendations
Action plan: to monitor and sign off the provider's action plan in response to
the recommendations.



Referring concerns to reviews
4
Where a higher education provider has a QAA review visit scheduled within the
next nine months, we may investigate the issues raised within that review and not produce a
separate report about it. If we choose to investigate through a review, we will pass the
information and accompanying evidence to the reviewers. We will explain the nature of the
concerns to the provider and invite them to provide a response to the reviewers.
The reviewers' subsequent view of the seriousness and validity of the concern may affect
the review outcome.
5
The Concerns Scheme is a reactive method and we are unable to predict when
concerns will arise. QAA handles concerns differently in review depending on the time they
are raised. This guide is structured around how to handle concerns arising at any time:



before the review visit
during the review visit
after the review visit (but before the publication of the review report).
1
6
In addition, some review teams will need to consider during the course of reviews
action plans related to concerns and provide advice to QAA on whether the action plan has
been successful completed.
7
This guide applies to reviews carried out under the following methods: Higher
Education Review, Higher Education Review: Wales; Higher Education Review (Alternative
Providers); Higher Education Review (Embedded Colleges, Higher Education Review
(Foreign Providers); Annual Monitoring for Educational Oversight; Annual Monitoring for
Specific Course Designation and Reviews of Transnational Education.
Concerns arising before a review visit
8
Through the process of screening and initial inquiry, the Concerns team may
identify evidence of potential systemic weaknesses that it should draw to the attention of the
review team. QAA will decide, depending on the level of risk indicated, whether to begin an
initial inquiry straight away, or to refer matters to the scheduled review, which may be up to
nine months away. In determining the level of risk, QAA will consider whether the concerns
represent low, moderate or high risk.
9
Low risks may relate to:



minor omissions or oversights
a need to amend or update details in documentation, where the amendment will not
require or result in major structural, operational or procedural change
completion of activity that is already underway in a small number of areas that will
allow the provider to meet the expectations of the UK Quality Code for Higher
Education more fully.
10
Moderate risks may relate to:


weaknesses in the operation of part of the provider's governance structure (as it
relates to quality assurance) or lack of clarity about responsibilities
insufficient emphasis or priority given to assuring standards or quality in the
provider's planning processes
quality assurance procedures are broadly adequate, but have some shortcomings
in terms of the rigour with which they are applied
problems identified are confined to a small part of the provision.
11
High risks may relate to:

ineffective operation of parts of the provider's governance structure (as it relates
to quality assurance)
significant gaps in policy, structures or procedures relating to the provider's
quality assurance
breaches by the provider of its own quality assurance management procedures
information in the public domain which is seriously inaccurate or has the potential
to mislead.





12
If the concerns represent a low risk, the Concerns team can refer the issues to the
next scheduled review visit if it is taking place within the next nine months. If the next
scheduled review is more than nine months away QAA should investigate the concerns
through the normal Concerns Scheme process.
2
13
If the concerns represent a moderate or serious risk then QAA will begin an initial
inquiry, in order to draw the matter to the provider's attention and gather evidence. At the
end of the initial inquiry, QAA will decide whether to:



close the case
refer any issues to the review team - either concerns or actions identified by the
provider to resolve issues
or begin a separate full investigation.
14
Issues that require urgent investigation or multiple, highly specialised or complex
concerns could justify a full investigation separate from the next review. Alternatively, review
teams could take on specialised or complex concerns by extending the visit to allow the
team time dedicated to the concerns, or by adding extra reviewers to the team.
15
QAA may refer concerns to a review team at any time in the months, weeks or days
before the review visit where a provider's review visit is scheduled within the next nine
months. QAA may consider bringing the review forward if the risks represented by the
concerns are substantial. QAA expects review teams and providers to be flexible in
accommodating what would become a new line of enquiry. QAA will write to the provider
advising that the review team will investigate the concerns. The letter will specify the nature
of the concerns and invite the provider to make a written response to the review team.
QAA will also inform those raising the concerns of this action.
16
In instances where the person making a submission to the Concerns Scheme asks
to remain anonymous, QAA will consider the feasibility of investigating the issues through
review while maintaining this anonymity. Protecting a person's anonymity might mean
evidence should be redacted.
17
The review team will respond to the concerns by evaluating the evidence provided
and identifying any additional evidence needed. Teams may also need to revise their
meeting agendas, and their list of key staff to meet on the visit.
18
The concerns referred to the review could lead to recommendations or affirmations,
and may have an impact on judgement areas.
Concerns arising immediately before or during a review visit
19
Where concerns become known to QAA immediately before or during a review visit,
the concerns may be investigated during the review visit. This may require an extension of
the visit in some review methods and we may add extra reviewers to the team. Alternatively,
QAA may investigate the concerns after the visit, using the process described below for
'Concerns arising after a review visit' (see paragraph 25).
20
The Concerns team will act quickly to screen submissions to the Concerns Scheme
that coincide with a review visit so that where possible issues can be referred and resolved
through review. In instances where the person making a submission asks to remain
anonymous, QAA will consider the feasibility of investigating the issues through review while
maintaining this anonymity. Protecting a person's anonymity might mean that evidence
cannot be shared with a team, or should be redacted.
21
QAA will ensure that concerns are evidenced before referring them to the review
team. QAA will not refer unsubstantiated allegations to review teams.
22
When QAA decides to refer concerns arising during a visit to a review team,
QAA will invite the provider to respond to the concerns during the meetings and support its
assertions with evidence.
3
23
It is possible that the investigation of concerns leads to recommendations or
affirmations, and may have an impact on judgement areas.
24
It is not possible to refer concerns to a review team that is already at its judgement
meeting, as the team has no opportunity to explore the issues with the provider. In such
cases, the concerns should be handled as 'Concerns arising after a review visit'
(see paragraph 25).
Concerns arising after a review or monitoring visit (but before
publication of the review report)
25
QAA will put the publication of review or monitoring visit reports on hold if concerns
arise after a review visit to allow them to be investigated. QAA will determine whether the
concerns have already been captured by the review team in their report, or whether they
represent new issues of which the team were unaware.
26
If the review team has taken account of the issues in their report, QAA will take no
further action.
27
If the concerns represent new issues that the review or monitoring team have
considered or reflected in their draft report, QAA will take forward the concerns through the
normal Concerns Scheme method, during which time the draft report will be put on hold.
Full investigation action plans
28
Following any concerns full investigation, QAA requests an action plan from the
provider setting out how it will meet QAA's recommendations. Occasionally providers share
action plans with QAA as the result of Initial Inquiries.
29
Normally QAA monitors completion of the action plan and signs it off. However,
when a provider is due a scheduled review or monitoring visit within nine months, the next
review or monitoring team may consider progress against the action plan. In such cases,
QAA should ensure the provider is aware of this and that the review or monitoring team
receives the Concerns full investigation report and the provider's action plan.
30
The review team will evaluate the completion of the action plan and formulate a
distinct statement about progress in completing the action plan for the report.
31
Ultimately, the review team need to check if the concerns action plan can be signed
off as complete, and if the team have confidence in the way the recommendations have
been met.
32
If actions are ongoing, the team would have to:

indicate whether the team is confident the actions taken are relevant and
appropriately address the recommendations
indicate if delays in the implementation of the action points are justifiable
indicate how much time would still be required to complete these before QAA can
sign off the action plan
consider whether to reiterate, reinforce or revise any recommendations in the
review report.



33
Action plans that cannot be signed off on the recommendation of review teams will
continue to be monitored by QAA's Concerns team.
4
QAA1633 - June 16
© The Quality Assurance Agency for Higher Education 2016
Southgate House, Southgate Street, Gloucester GL1 1UB
Registered charity numbers 1062746 and SC037786
Tel:
01452 557050
Website: www.qaa.ac.uk
5