- The TLQPR process begins with a preliminary visit by the Panel
to each institution for the purposes of familiarizing staff with
the purposes and methods of the Review, and the preparation by
the institution of a twenty-page document describing its quality
improvement and assurance processes. The Review visit lasts one
and one-half days, which are utilized as follows:
- The first half day is devoted to three meetings: with the
institutionís senior leadership, with the leadership plus
academic staff associated with the quality improvement and assurance
programme, and with students.
- The second half day involves meetings at the faculty level
or with academic departments or quality programme support units.
The Panel divides itself into six sub-groups for this purpose.
Each subgroup meets with academic staff, students, and the leadership
from two operating units, which allows visits to twelve units
in all.
- The third half-day begins with a private session where the
Panel formulates its preliminary impressions about the visit.
The visit ends with a final meeting with the leadership and staff
involved in quality assurance, where the preliminary impressions
can be conveyed and discussed.
- The TLQPR Panel consists of eighteen people: nine (including
the Chair) members of the UGCís Quality Sub-Committee;
two non-UGC overseas members with experience in higher education
quality assurance; and one member from each of the seven Hong
Kong institutions and starting from September 1996, one observer
from the Hong Kong Institute of Education. The local members were
designated by a larger TLQPR Consultative Committee of local-institution
representatives, which has assisted the UGC in designing the TLQPR
methodology. Because the TLQPR is collegial, the local Panel members
participate fully in the reviews of their institutions.
Report Preparation
- Report preparation proceeds in several stages. First, the institutionís
self-analysis and discussion notes from the early plenary sessions
are scrutinized for emergent themes and examples of exemplary
and questionable practice. (The self-analysis summary uses the
institutionís language to the extent possible.) The subgroup
reports are similarly scrutinized, and a summary is prepared.
The draft of this part of the report is reviewed by the Panel,
and then by the institution for factual accuracy before submission
to the UGC's Quality Sub-Committee. The "Areas for
Improvement" section is drafted concurrently and reviewed
by the Panel, the Quality Sub-Committee, and the UGC before the
final Report is transmitted to the institution. The institutions,
in turn, have committed to make the reports public along with
a statement describing the actions they plan to take by way of
improvement.
TLQPR Dimensions
- Teaching and learning quality can be viewed from two different
perspectives. First come the teaching and learning processes themselves;
in other words, the activities performed by academic and support-unit
staff in performing their duties. Second come the methods by which
institutions, faculties, departments, and similar units work to
continuously improve teaching quality and assure themselves that
the activities are appropriate and well executed.
- The Review Panel recognizes that decisions with respect to
both quality perspectives must be made by the institutions
themselves, and that variety among and within institutions is
necessary for an effective tertiary sector. The Panelís
fundamental standard, therefore, lies not in specifying any particular
approaches to teaching and learning quality, but rather in asking
whether institutions and academic staff have given careful thought
to both of the quality dimensions and whether they can articulate
and defend the choices made.
Teaching and Learning Processes
- Teaching and learning processes can be described in terms of
the following five sub-processes, which form one dimension of
the Panelís inquiry. Each sub-process is illustrated by
questions which might be asked of an institution, a faculty, a
department, or an individual staff member. However, the questions
are presented by way of example only. The Panel does not presume
that all the questions, or indeed any of them, are applicable
in any particular situation. However, we do ask the institutions
to organize their documentation in terms of the five sub-processes
and we refer frequently to the five in our deliberations.
- Curricular design: by what processes are programme curricula
designed, reviewed, and improved? Some useful process elements
follow:
- Design inputs from the academic discipline, mainly staff-based
- Design inputs from employers, feedback from current outcomes
assessments, past students, professional bodies (where applicable),
and other inputs dealing with ìfitness for useî
- Integration mechanisms: how are these two kinds of inputs
brought together? How are controversies resolved?
- Faculty and institutional review mechanisms; what are they
and how do they work?
- External review mechanisms; e.g., visiting committees
- Pedagogical design: by what process are the methods
of teaching and learning decided and improved?
- To what extent are pedagogical methods the subject of active
consideration by staff, departments, faculties, etc.? Do staff
spend sufficient time working together on these matters?
- How broad is the definition of ìpedagogical methodî?
For example, does it focus on learning as well as teaching? Does
it integrate feedback about learning attainment with the delivery
of academic content?
- Degree of innovation in pedagogical method? Have the methods
been changing over time? For example, have they been trending
toward active as opposed to passive learning? Have they been taking
sufficient advantage of information technology?
- Implementation quality: processes related to how well
the staff perform their teaching duties
- How broad is the definition of ìteachingî? Does
it include out-of-class student contact (including advising) and
student assessment (including feedback about the assessments)
as well as class contact?
- What are the incentives for good teaching? What are the disincentives?
(It is important to consider staff perceptions as well as the
programmes themselves.)
- How is teaching performance evaluated? (Possible mechanisms
include self-evaluation, student evaluation, and peer evaluation.)
- How are teaching evaluations utilized? For example, are they
used in staff evaluation reviews? Are they shared among staff
as part of a mutual-improvement process? Do they result in specific
self-improvement efforts, such as utilization of teaching improvement
centres?
- Outcomes assessment: how do staff, departments, faculties,
and the institution monitor student outcomes and link outcomes
assessments to teaching and learning process improvement?
- Academic performance: for example, normed examinations and
the use of external examiners
- Other performance; for example, satisfaction as expressed
in exit conferences, success in the job market
- Feedback from past students, employers, etc.?
- Are processes for working with students to help them achieve
the desired teaching and learning outcomes in place and fully
functioning?
- Resource provision: are the human, technical, and financial
resources needed for quality made available when and where needed?
- Are the activities needed to achieve and assure teaching and
learning quality given an appropriately high priority in the institutionís
resource allocation process?
- How do staff recruitment processes promote and safeguard the
quality of teaching and learning?
- How does the institution's incentive and reward environment
further the teaching and learning quality agenda?
- To what extent does the institution offer technical assistance
and training to staff who wish to improve their teaching and learning
quality performance? To what extent are these resources utilized
by staff?
Quality Improvement and Assurance
- The Panel does not approach its task with any preconceived
view of what an appropriate quality improvement and assurance
programme should look like. On the contrary, we emphasize that
the institutions should define their own processesóthat
the Panelís job is to see whether such processes have in
fact been defined and, once defined, whether they are being followed
diligently. This view is consistent with the emergent international
understanding of teaching quality in tertiary education, and with
the fact that universities in Hong Kong are self-accrediting.
- Certain broad areas of consideration for successful quality
assurance have emerged from the Panelís queries and discussions,
and these are presented below. We use them to convey examples
of potentially useful quality assurance and improvement methods
and to provide an organizing paradigm for our reports, but not
as a template for judging an institutionís quality programme.
However, the Panel does believe that to be fully effective, the
assurance and continuous improvement of quality require a degree
of self-consciousness and articulationówhich should be
observable in the Review documents and site visit.
- Quality programme framework: mission, vision, and policy
statements pertaining to quality and quality assurance, expressed
at the institutional level and at the level of faculties, departments,
and other operating units. The framework provides a road map for
individual and group action aimed at furthering and assuring teaching
and learning quality.
- Direct quality programme activities: undertaken by
mainline teaching and administrative staff at the institutional
level and at the level of faculties, departments, and other operating
units. These activities are organized to assure quality levels
and continuous quality improvement in the teaching and learning
activities described in paragraph 6.
- Quality programme support: funded special projects
or activities undertaken by special teaching development or similar
units organized to aid mainline teaching and administrative staff
in performing their duties.
- Values and incentives: the motivational environment
for the improvement and assurance of teaching and learning qualityódriven
by institutional, faculty or departmental values (intrinsic rewards)
and formal or informal incentives (extrinsic rewards).
Sample Approaches
- The matrix presents some examples of how the four
quality improvement and assurance methods can be applied to the
five teaching and learning process dimensions. We observed each
example in at least one of the institutions we visited, either
centrally or at the level of a faculty, department, or other operating
unit. The examples illuminate our framework for the TLQPR, but
they are not intended to be definitive or prescriptive. Discussion
of the various items and observations relevant to the individual
institutions we visited is contained in the body of the report.
**********