| Annex C |
The Unit is very much course focussed and course driven, and it
appears to lack clear objectives or a clearly defined mission.
We found it difficult to see where broader policy debate takes
place -- although course and subject leaders meet on a regular
basis (for operational purposes) and there had recently been a
departmental retreat.
We met a very close knit Unit with strong leadership. They are
working very effectively toward high-quality in teaching and learning
using a range of well-executed and highly internalised processes.
They have a well established tradition and culture of teaching
quality and attention to studentsí learning. While formal
processes were present, their primary strength was based on strong
informal interactions. While the teamwork and results seem to
be excellent, we feel that the system could be more proactive
and be more robust if its processes were more formalised. There
was little mention of Faculty or University influence or procedures;
they apparently felt that their internal systems were fully satisfactory
and owned by the department, which indeed they appeared to be.
On the other hand it was not clear if they were coupled into the
rest of the university or Faculty enough to help other units or
to learn from them.
The Unit appeared to be tightly managed and the Head stated that
the emphasis now is on ensuring greater ownership of activity
amongst colleagues, on greater transparency in the decision making
process and on the development of collegiality. Assurance processes
had evolved from being top-down, with stringent course validation
requirements, to more departmentally based and open processes
with an emphasis on enhancement and greater staff ownership, communication
and accountability.
The Faculty has an excellent policy document on teaching, and discussions of teaching at Faculty level are lively and interesting. However, translation to departmental actions appears to be optional and variable. For example, the Faculty favours the idea of using some peer review of teaching, but this is carried out in only one department -- it is left to each head to decide. The Faculty also now recommends but does not require the submission of teaching portfolios with promotion cases. Policies on the development of staff whose teaching is weak also are left up to departments.
Although the Annual Course Review is considered by a Faculty Board
Working Party and some comments are received on the action plan,
skepticism was expressed about the value added of Faculty-level
involvement. At departmental level, however, the processes appeared
effective. There is significant external feedback, which is evidently
acted upon, and there also is an active staff/student Consultative
Committee which operates on the basis of shared agendas and with
the minutes jointly agreed with the students. The Student Feedback
Questionnaire is used within the Unit as a developmental tool.
There is also a course appraisal by final-year students. Actions
designed to promote a quality culture include revised committee
structures involving elected members and greater transparency
of decision-making.
Programme quality assurance processes are in place, but there
appears to be a lack of co-ordination among "common core"
subjects -- perhaps because the structure is complex due to the
wide array of courses. Follow-up action after approval of the
Annual Course Review report by Faculty Board seems to be dealt
with in a rather ad hoc manner. No formal mechanism could
be discerned. The Faculty Board merely leaves "it to the
Chairman to form a small group, with one nominee from each Department,
to consider the findings of the Report". Given the complexity
and plethora of courses, the effectiveness of the "small
group" approach seems doubtful.
The staff have an impressive and strongly internalised culture
of quality. They showed belief in continual review and improvement.
However the connection to the Faculty seemed weak and in some
future time of change the continuation of their excellent informal
processes might not be ensured by the existing weak Faculty and
University oversight. This is a Unit with a quiet but evidently
growing commitment towards quality in teaching. No doubt the process
could gain even further momentum from a more explicit and conscious
approach.
Staff express positive attitudes about teaching quality, but their
attitudes toward quality assurance are distinctly less positive.
They seem to think that this kind of exercise (TLQPR) does not
provide useful insights for those who teach or invest resources
in teaching. Probably the Unit's quality is good, but there
is no strong or visible commitment to mechanisms for stressing
and improving teaching quality.
The Unit's policy is that teaching is the basic function
of a staff member. The evidence presented by a staff member for
promotion is in the form of student questionnaires and reports
from course leaders. One staff member expressed concern about
the increased attention on teaching quality assurance procedures.
He believes such procedures are unnecessary. As he put it, "We
teach because we like to teach!"
We were provided with a thick volume of documentation consisting
of validation reports, course definitive documents, course committee
meeting minutes etc. It appears that formal procedures are in
place in respect of course design, validation, course monitoring,
obtaining feedback from students, graduates and employers and
so on. These procedures have apparently been in place for some
years, following the polytechnic tradition of rigorous course
validation by CNAA and HKCAA. (Revised staff development, staff
appraisal and student evaluation policies were introduced three
years ago, after obtaining university status.) It was quite obvious
from our discussion with the staff that they could not see quality
assurance beyond the formal processes and there was little attempt
to cultivate a quality culture. The staff seemed to take for granted
the top-down approach, always waiting for the administration to
give instructions on quality assurance matters.
Course validation and revalidation are vetted by the Faculty.
This seems to be the only procedure in place for across-faculty
teaching and learning monitoring and evaluation and reaction to
results. The Faculty has agreed that teaching "learning
how to learn" is important to incoming students. The Dean
and at least one other staff member teach this to incoming students.
Curriculum design is primarily course-driven based on input from
strong employer networks. Problems with common core and service
courses were identified, but felt to be largely outside the Unitís
control.
The Unit appeared to have a well articulated system involving
subject committees, course committees, staff and students as well
as departmental advisory committees and academic input from external
examiners.
Close contacts are maintained with local businesses and government
and enthusiasm was seen for updating the curriculum to keep up
with changes in the profession, increased interactions with China,
etc.
The Unit has a very elaborate curriculum design with several external and internal actors. The relative role of external and internal feedback is supposed to depend on the context and perspective, but it appeared that most design issues were decided upon on the basis of internal planning and priorities. The Unit has considerable experience with professional accreditation, but does not see this as having a real impact on quality strategies.
The Unit's feedback structure is generally satisfactory, and it
makes especially good use of contact with alumni associations.
On the other hand the subgroup met the usual problem of not being
able to find visible mechanisms for adapting and giving priorities
to external feedback. In terms of curriculum design, academic
ideas go first.
Pedagogical methods are primarily matters for the course leader
and individual staff. Informal discussion among colleagues may
well occur, but there was no evidence of active debate on pedagogical
design or the implications for teaching and learning of changes
in the student intake profile in recent years.
Pedagogical methods appear to be the subject of active discussion
at subject and course level both formally in committees and more
informally in Faculty workshops and seminars, and daily peer-group
interaction.
Staff emphasize informality of contact and interactive learning
by discussion and conversation; and we noted that their teaching
space has been arranged to support these objectives. The Unit,
we were told, is committed to small group teaching which research
has shown to be particularly important in their discipline. Thus
class sizes are kept at 20 (as against an institutional expectation
of 40) in their service teaching; and this, together with the
small size of cohorts in their 'content' courses, means that they
are unable to benefit from scale economics of balancing small
groups and large lectures. The students we met confirmed that
they have good informal contacts with their teachers but regretted
the fact that most of their courses are compulsory, leaving them
little freedom of choice.
The Unit has two projects on multimedia with the EDU, but among
staff members present some had a relatively small interest or
knowledge of EDU and its services while a few staff members had
a more positive experience. EDU services were approached and used
on a very individual basis. This individualized attitude towards
quality teaching could be either the cause or the effect of the
lack among staff of a common framework of reference for the enhancement
and measurement of quality in teaching.
Implementation Quality
The professional training of many members of staff makes for considerable
sophistication in matters of teaching and learning quality. As
a policy the Unit has moved to employing instructors only on a
full-time basis, in order to be able adequately to monitor their
work, and to promote their development. The Unit intends to formalize
the induction of new staff, though informal monitoring would still
be important: 'We talk about teaching an awful lot'. The Universityís
formal appraisal mechanisms are held to have been imposed without
consultation, and to be top-down (as opposed to a peer review
or upward appraisal). However, they are now being embedded in
the Unit. Fourteen appraisers have been trained, which means that
in future they will each appraise five or six members of staff.
Other mechanisms include teaching portfolios, action research,
observation and recording, between-semester workshops, and retreats.
The Unit, we were told, works with the EDU on a quid pro
quo basis. Student evaluation is again both formal and
informal; and the students we met felt that their opinions, made
known individually or through class representatives, were taken
note of, and that their feedback was effective.
Student feedback appears problematic: course consultative meetings
seem to be used to give out messages to students rather than to
hear what they have to say. The semi-annual course committee meetings
were seen by staff as the forum where a student view could be
put, but students said they lacked information on the nature of
courses or on proposed changes, they were not clear about expectations
of their contribution and doubted that their views would be acted
upon. Representatives on Course Committees did not appear to report
back to any wider student group. Whilst staff acknowledged that
student feedback is a problem, there does not seem to be any proposals
to improve things. We concluded that the Quality Assurance processes
were effective at the level of the course, but we were unsure
of the contribution made at Faculty or University level. Neither
the department nor the Faculty appeared as cohesive academic entities
with clear roles. The Student Feedback Questionnaire is treated
by staff and students solely as staff appraisal instrument. Course
evaluation questionnaires are used, but apparently not systematically.
The standard Student Feedback Questionnaire and course evaluation
questionnaires are used proactively for both staff development
and staff appraisal purposes. The departmental culture has accepted
the need for educational programmes for new staff and continual
improvement for existing staff. Some use is made of the services
of the Educational Development Unit.
The students are in general happy with what they receive from
the different courses. They are, however, dubious whether the
feedback they provide by filling out the course and teaching evaluation
questionnaires at the end of each course is actually taken into
account by staff. Although student representatives sit on committees
that deal with policy issues, there is no staff-student consultative
committee where they can express their views on teaching and learning
in specific terms.
The Student Feedback Questionnaires (SFQ) are mandatory. They
are used with two classes each year and are the main basis for
staff appraisals. The Course Evaluation Questionnaires (CEQ) and
the Subject Evaluation Questionnaire (SEQ) sponsored by the EDU
are voluntary and provides the individual teacher with a feedback.
Staff experience with these were varied and not too substantial.
But there is a departmental policy of encouragement for attending
the EDU courses in teaching quality.
Staff-student consultative meetings function well. Staff as well
as students were seemingly in agreement that informal processes
are more effective and constructive than the quantitative approach
to quality in teaching on which questionnaires are based. The
system where all teachers meet with their students after midterm
and students from all three years are involved in the proctoring
system is seen not so much as a monitoring system as a de facto
safety net for students.
The Department is located within a Faculty which, according to
the staff, is a diverse mix of departments. In administrative
terms, the Department described the Faculty as largely a quality
assurance framework, with resource or budget control mainly in
the areas of research and staff development funding. The Head
welcomed the move towards Departmental Assessment as a natural
progression, and accepted the responsibility of the Department
to deliver quality, although feeling that - while the Department
was not understaffed - there was insufficient middle management
to achieve this assurance satisfactorily.
Feedback from students, alumni and employers is actively sought
and used in annual course review and periodic course revalidation.
Regular questionnaires provide feedback from employers, which
indicates that graduates from the Unit are practical and able
to adapt to a working environment -- a view also voiced, as self-perception,
by the students we met. Staff also consider the rise in applications
for their programmes as good indicators of quality. The exit test
for the language proficiency of graduates, stipulated in the University's
Policy Paper, was generated in the Department as the result of
a major, still on-going, research project on the language abilities
of graduating students.
The Unit indicated a largely traditional reliance on external
examiners and professional recognition. Feedback from employers
and alumni was also sought via workshops with industry participation.
Strong involvement in professional societies, employers, etc.
and attention to graduatesí and employersí opinions.
External examiners are not presented as having any strong operational
role.
The Unit has complete control of the resources for teaching their
course. They have the power to decide to pay another department
to teach a subject for them or to hire someone themselves. They
have used this power to have a lecturer changed in a subject taught
by another department. The Unit has a well developed set of policies
on staff and subject development, including staff doing research
in areas relevant to teaching, Ph.D. and other study support,
conference and seminar attendance, etc.
Essentially, the Unit has no resource responsibility.
Staff expressed some concerns about dedicated space and equipment
for their teaching. Asked about time for research, they answered
: 'not enough'. They found it difficult to quantify the balance
of teaching and research, and felt that administrative duties
encroached on both. They also declared themselves 'bedevilled'
by the distinction in scales between 'Polytechnic' and 'University'
staff. Their research, they emphasized, is often 'focussed on
what goes on in the classroom'; and their research interest and
expertise are to an extent deployed by the University for quality
enhancement.
There is tension among some teaching staff as to what constitutes
a right balance between research and teaching. Although the Faculty
emphasizes that teaching, research and publication, as well as
service are important in staff review, there is lack of evidence
that good teaching is actually rewarded.
The Unit welcomes the greater freedom provided by its one-line
budget. Promotion and other institutional incentive systems are
perceived as based heavily on research despite strong departmental
emphasis on teaching and learning.
The Educational Development Unit was formed in 1995 following
the restructuring of the former Educational Technology Unit into
three separate units, of which the other two are dedicated to
Audio-Visual and Materials aspects of educational development,
whereas the unique function of the EDU, in the words of its Head,
is 'working with people'.
The Head reports to the Vice President for Quality Assurance,
and the EDU operates within the University's Strategic Planning
for Assurance and Enhancement of Learning and Teaching (SPAELT).
Its function, we understand, is to seek to enrich students' learning
experience through the use of learning-centred teaching methodologies
and new educational technologies, and to support this endeavour
by the design, development and delivery of professional education
and training programmes.
The policy contribution made by the EDU is the Strategic Planning
for Assurance and Enhancement of Learning and Teaching.
The EDU also gives professional support to the Learning and Teaching
Development Committee (LTDC: established in 1996 as a committee
of Senate), which has responsibility for the enhancement of teaching
quality by academic staff including the awards of Teaching Development
Grants. The Head of the EDU is also an observer on the Academic
Quality Assurance Committee.
Services offered by the EDU are set out in a comprehensive Guide.
They include (1) a number of workshops, such as an intensive two-day
programme for staff new to teaching; an extended, ten-week, 'Introduction
to University Teaching'; lunchtime discussion sessions; workshops
for Teaching Assistants; and tailor-made workshops for departments;
(2) support for the evaluation of courses and teaching, through
Student Feedback Questionnaires (SFQ), Course Evaluation Questionnaires
(CEQ) and Subject Evaluation Questionnaires (SEQ); and (3) development
of materials, of World-Wide-Web teaching (through the Electronic
Educational Development Centre), and of multimedia teaching.
The unit also offers a self-funded, part-time Postgraduate Diploma/MEd
in Teaching in Higher Education.
The agenda of the EDU is largely set by the staff themselves:
the LTDC is mainly concerned with Teaching Development Grants,
and there is no steering group or user group. We were told that,
in recent University QA developments, such as the several Policy
Papers and the Departmental Assessment scheme, the EDU has been
consulted, but not systematically. To link the unit with university
staff, there are departmental liaison arrangements, as well as
individual consultancies with members of staff.
The EDU has its own Quality Assurance system (and under the new
University Generic Quality Assurance scheme, this will have to
be formally presented and approved). It comprises feedback from
participants in all workshops and other programmes organised by
the unit. Systematic feedback is gained by visits to all Heads
of Departments and by a questionnaire survey of all staff. There
is no formal feedback from students. For teaching QA in respect
of its Postgraduate Diploma/MEd programme, the unit is deemed
to fall within the Faculty of Health and Social Studies. The EDU
is similarly responsible for its own staff development. This,
we were told, includes encouragement of conference participation,
of work for higher degrees, and liaison and interactions with
other units.
Management within the unit is largely informal, with continuous
contact and sharing of initiatives and problems between members
of staff. Formally, the unit meets periodically but infrequently.
In our discussion with members of staff of the unit, we were impressed
by their enthusiasm and their vigorous commitment to the cause
of improving teaching and learning quality in the institution.
The EDU represents a fresh start for the University in the area
of promoting TLQ through staff activity, professionalism and continuous
improvement. The unit's strategy is pro-active : members of staff
repeatedly stressed to us that they did not wish to be seen as
a 'teaching police', and that their workshops and the practices
they are promoting - in feedback, appraisal, teaching portfolios,
etc. - are not mandatory but voluntary. Because its policy is
problem-based and student-centred, the unit does not focus on
skills but on creating a climate and culture of teaching and learning
quality. Its function, we were told, is above all to stimulate,
and it is in keeping with this that its activities are largely
initiated by individual contacts.
There are great strengths in this strategic framework and in the
excellent projects promulgated by the EDU; but inevitably there
are also weaknesses. As yet - and of course it is early days -
evidence of the impact of the EDU on the institution is a little
patchy; and there is the danger of the unit going too far in the
direction of activity which is focussed and based on research,
rather than on ensuring ownership of quality throughout staff.
We felt quite strongly that, while the EDU is a laudable initiative,
and while its staff is doing excellent work, the unit's potential
is not sufficiently realized and utilized by the institution as
a whole. The unit needs to be integrated more firmly to central
University policy and committee structure. It also needs to have
a user/steering group which will help create a focus on overall
ownership of the quality culture which the unit as such embodies.
________________________________
* Paragraphs in this Annex generally relate to different units.