 |
 |
Teaching and Learning Quality Process Review - A Review of the Hong Kong Programme1 |
|
|
|
William F Massy
Professor: Stanford University
&
Nigel J French
Secretary-General, University Grants Committee of Hong Kong
Contents
|
Introduction
| 1.1 |
Hong Kong's higher education system is highly developed, on lines that will be familiar to anyone with a knowledge of tertiary education in the Commonwealth tradition, and very diverse. All but one of the nine degree-awarding institutions are predominantly publicly funded. The local age participation rate in higher and higher vocational education is about 25%. On top of that it is estimated that perhaps as much as another 10% of the age cohort pursue higher education overseas, principally in North America, Australia and the UK.
|
| 1.2 |
A major report published at the end of last year by the University Grants Committee of Hong Kong (UGC) described the present landscape of higher education in Hong Kong in detail:
"Taking students as the descriptor, we have 40,000 on full-time first degree courses in Hong Kong and a further 28,000 on full-time undergraduate courses overseas. In addition, there are some 16,000 fte students studying part-time for first degrees, half of them through the Open Learning Institute of Hong Kong (our open university). In the sub-degree area, 22,000 fte students are taking higher diploma or other high level courses, 2,500 fte are studying for higher certificates or their equivalents and 1,000 fte for a variety of other qualifications. Some 6,700 fte students are engaged in postgraduate work in Hong Kong, either on taught courses or as research students, and there are a further 12,000 overseas. The largest category of those enjoying higher education is, however, the 46,000 fte students in CPE (continuing and professional education). About half of this number are working in UGC institutions and half in other subvented or private colleges or in-house at their employers."2
|
| 1.3 |
This extremely rich and diverse picture is a relatively recent creation, however. As little as 15 years ago the local age participation rate in an avowedly elitist system was below 5%; in 1990 it was still below 15%. The significant expansion of opportunity for Hong Kong school leavers to partake of tertiary, particularly degree-level, education in the Territory is not the subject of this paper, but it is an important part of the background - see below.
|
| 1.4 |
The UGC, of which Prof Massy is a member and Mr French is the Secretary-General, is a non-statutory advisory body whose members comprise distinguished overseas academics, prominent local professionals and business people, and senior locally-based academics. It has the responsibility of advising on the academic development and funding of Hong Kong's institutions of higher education. It also plays a vital role in assuring the quality of higher education provision in the tertiary education institutions under its aegis.
|
| 1.5 |
As part of the UGC's on-going activities to assure quality and value for money in the higher education sector, the UGC embarked in 1995 on a programme of teaching and learning quality process reviews. This followed the award of self-accrediting status (and subsequently university title) to two former polytechnics and one liberal arts college in 1993 as well as the implementation of a performance-based funding model and a research assessment exercise in 1994. This paper describes the background to, and methodology of, the "Teaching and Learning Quality Process Reviews" (TLQPRs) and summarises the outcomes of the first round of TLQPRs of seven UGC-funded tertiary education institutions. It also describes the lessons learned from the programme, how they are expected to be followed up by the institutions and the UGC, and presents ideas for change that the UGC might consider in the next round. The theory behind adoption of the TLQPR approach and an early description of the methodology were presented in a previous paper.3
|
|
|
Background
| 2.1 |
During the five year period from 1991 to 1995, Hong Kong achieved a rapid expansion in its higher education sector. Student enrolments (in FTE terms) at the UGC-funded institutions increased by nearly one-third from 1990-91 to 1995-96. There was also a doubling of the number of first year first degree places, a 66% increase in the number of first degree students, and a 123% increase in the number of postgraduate students. As a result, many new courses and more students of a wider range of ability had to be taught, and more academic staff were required to teach at degree and postgraduate levels.
|
| 2.2 |
Following this period of rapid expansion, when the immediate public policy objectives in terms of increased access and more higher level education and research had largely been achieved, the UGC saw as a major challenge the maintenance of academic standards in general, and the quality of teaching and learning in particular. It was against this background that the UGC decided, as part of its evolving approach to the assurance of quality in the UGC-funded institutions, to undertake TLQPRs of all the seven institutions then under its aegis. A further impetus was the achievement of self-accrediting (and subsequently university title) by two former polytechnics and one liberal arts college in 1993, when the UGC recommended and the Government agreed that the UGC should undertake periodic process reviews of all the UGC-funded institutions.
|
| 2.3 |
The TLQPRs were undertaken, during an 18 month period from September 1995 to April 1997, by a specially constituted TLQPR Panel consisting of eight UGC members, fourteen members from the UGC-funded institutions and two overseas experts on higher education quality assurance. All members (or their designated alternates) participated in all the Panel's activities, including visits to their own institutions. With effect from September 1996, following the designation of the Hong Kong Institute of Education (HKIEd) as a UGC-funded institution in July 1996, a representative from the HKIEd was invited to participate in the work of the TLQPR Panel as an observer.
|
| 2.4 |
The use of such a large panel, which is a major difference between the TLQPR procedure and that commonly used in similar audits in other countries, was deliberately designed to encourage mutual learning and acceptance of the process locally. As indicated earlier, fourteen of the Panel members, including four of the UGC members, were locally based academics. This it undoubtedly achieved, but arguably at the expense of some consistency and understanding in the Panel's deliberations.
|
| 2.5 |
Throughout the planning, preparation and implementation stages of the TLQPRs, the Panel and the UGC were greatly assisted by a Consultative Committee made up of representatives of all the UGC-funded institutions being reviewed in the first round and, with effect from mid 1996, the HKIEd. The existence of this Consultative Committee, which also took the lead in the organisation of a seminar in April 1997 to discuss the impact and share some of the lessons learned in the first round of TLQPRs (see below), also contributed significantly to understanding and acceptance of the process in the local institutions.
|
| 2.6 |
The goals of the TLQPRs are to focus attention on teaching and learning, to assist institutions in their efforts to improve teaching and learning quality, and to enable the UGC and the institutions to discharge their obligation to maintain accountability for quality. These goals are shared by the UGC and the institutions. To emphasise the importance of teaching and learning, the UGC has announced that the TLQPR results may be used to inform funding decisions.
|
| 2.7 |
A TLQPR of one institution takes approximately eight months starting from a preliminary visit by the TLQPR Panel to the institution for the purposes of familiarising staff with the purposes and methods of the review, and the preparation by the institution of a twenty-page document describing its quality improvement and assurance processes. The review visit lasts one and a half days which are utilised as follows:
the first half day is devoted to three meetings: with the institution's senior leadership, with the leadership plus academic staff associated with the institution's own quality improvement and assurance programme, and with students; the second half day involves meetings at the faculty level or with academic departments or quality programme support units. The Panel divides itself into six sub-groups for this purpose. Each sub-group meets withacademic staff, students, and the leadership from two operating units, which allows visits to twelve units in all; and the third half day begins with a private session where the Panel formulates its preliminary impressions about the visit. The visit ends with a final meeting with the leadership and staff involved in quality assurance, where preliminary impressions are conveyed and discussed.
|
| 2.8 |
Report preparation proceeds in several stages. First, the institution's self-assessment and discussion notes from the early plenary sessions are scrutinised for emergent themes and examples of exemplary and questionable practice. (The self-assessment itself is published as an annex to the Panel's report.) The sub-group reports are similarly scrutinised.
|
| 2.9 |
In the case of the first two institutions reviewed, the draft of this part of the report was reviewed, after initial consideration by the Panel, by the institution for factual accuracy before submission to the UGC's Quality Sub-Committee and the UGC. The "Areas for Improvement" section was drafted concurrently and reviewed only by the Panel before submission to the Quality Sub-Committee and the UGC and before the final report was transmitted to the institution. In subsequent reviews, the full reports (including the "Areas for Improvement" section) were reviewed in draft, after initial consideration by the Panel, by the institutions concerned for factual accuracy before submission to the Quality Sub-Committee and the UGC.
|
| 2.10 |
The reports of the reviews, once finalised and forwarded to the institutions concerned, became their property. However, all the institutions have committed to make the reports public, along with a statement describing the actions they plan to take by way of improvement.
|
| 2.11 |
At the time of writing this paper, TLQPRs of all the seven institutions have been completed. Five reports have been published, together with the statements of initial response of the institutions concerned; the other two reports have been finalised and forwarded to the institutions concerned. They are expected to make them public within the next few weeks.
|
| 2.12 |
As part of the process of evaluating the TLQPRs as a whole, the UGC sponsored a seminar, organised by the Consultative Committee in April 1997, involving the TLQPR Panel members, the Consultative Committee and some 150 other colleagues from the institutions to discuss the TLQPR programme, the lessons learned and the way forward. This paper takes into account comments arising from that seminar.
|
|
|
Dynamics of the TLQPR Process
| 3.1 |
Throughout the period that the TLQPRs were being undertaken, the Panel, the UGC and the institutions being reviewed were feeling their way with the process. This mutual learning aspect was also emphasised in discussions with the Consultative Committee, which acted as a useful forum for discussing new ideas and suggestions emanating from the Panel or the institutions.
|
| 3.2 |
Following the experience of the first two reviews (of the University of Hong Kong and the Chinese University of Hong Kong), the Panel consciously sought to place more emphasis on the "structured conversations" in the unit-level meetings. Department and faculty heads were encouraged to make shorter introductory presentations, thereby leaving more time for discussion and active involvement of other faculty and students. In practice it still tended to be difficult to involve students actively in the full unit-level sessions. The separate meetings between the sub-group members and the students were rather more useful in this regard, but still often did not get beyond superficial issues. For the future, there will be a need to reconsider the best means of obtaining student participation and feedback in the TLQPR process.
|
| 3.3 |
In terms of institutional documentation, the Panel in most cases found it necessary to request the institutions to provide more in the way of self-assessment. This was because the institutions tended initially to be insufficiently precise and explicit as regards the strengths and weaknesses of their existing quality assurance processes. As indicated earlier, the practice developed of requesting the institutions to provide a separate self-assessment document for inclusion as an annex to the Panel's own report. The institutions' self-assessments were, at the Panel's request, presented in terms of the five sub-processes which formed one dimension of the Panel's inquiry viz. curriculum design, pedagogical design, implementation quality, outcomes assessment and resource provision. Since self-assessment probably represents the major learning experience for the institutions, such self-assessment reports could well serve as the sole, or at least principal, documents for the purposes of the next round of TLQPRs.
|
| 3.4 |
At the unit level, the quality and level of detail provided by different departments/faculties visited was varied, but in itself often revealing. While not wishing to suggest that documentation is the end of quality assurance, there was often a close correlation between the clarity and conciseness of the documentation provided and a department's or faculty's understanding of and engagement in the formal and informal processes of teaching and learning quality assurance.
|
| 3.5 |
The unit-level meetings and subsequent sub-group reports (initially orally to the Panel as a whole and subsequently in writing) are central to the TLQPR process. The discussions that developed between the sub-group members and the staff and students of the units visited in just two hours were generally of remarkably high quality. Much depended on the ability of the sub-group convenors and the unit heads to encourage a free-flowing and open exchange of views, but normally the right atmosphere was able to be achieved quickly. All members of the sub-groups contributed to the subsequent reporting and analysis of what had been seen and heard (and what had not been seen or heard, which was often equally or more important), but the sub-group convenors had the ultimate responsibility for writing up the sub-groups' findings.
|
| 3.6 |
During the course of the seven TLQPRs, the Panel developed a strong 'esprit de corps'. No distinction was made between members based on their institutional backgrounds--the HKIEd observer was welcomed and treated as a full participating member of the Panel from the start.
|
| 3.7 |
The role of the Panel Chair was crucial. For this first round of TLQPRs, the Panel Chair was also the architect of the TLQPR process within the UGC--Professor Massy having been actively involved as Chairman of the UGC's Teaching Quality Sub-Group and member of the UGC's Quality Sub-Committee from its inception in 1994. He had also had considerable experience with other UGC quality assurance activities including the Sectoral Reviews in 1992 and 1993 and service as chair of the first Research Assessment Exercise in 1993.
|
| 3.8 |
The Panel Chair had three principal functions:
to lead and maintain cohesiveness among the Panel members; to assimilate rapidly the oral reports of the sub-group convenors and synthesise the messages to be conveyed, initially orally, in feedback to the institutional management; and to produce the first draft of the report on each TLQPR and be responsible for overall consistency of style and presentation in the final reports.
|
| 3.9 |
It is clear that the selection of the right person to chair the Panel is critical to the success of the TLQPR process as currently designed. For future exercises, the Panel Chair may, however, need more support in the drafting stage, that is in terms of academic input, not just secretarial support.
|
|
|
Summary of Review Findings
| 4.1 |
While a detailed description of findings for the seven institutions is beyond the scope of this paper, we can offer a brief summary. The TLQPR served to reassure the UGC that teaching and learning quality processes in the tertiary institutions are broadly satisfactory. Most important, the quality assurance and improvement processes appear to be getting better, partly as a result of the TLQPR's focusing attention on the subject. The Committee hopes that publication of the remaining reports will further promulgate best practice and stimulate a commitment to even greater improvement.
|
| 4.2 |
The TLQPR reports identified a number of policies and practices as being particularly noteworthy in the context of the Review's agenda. They are summarised below in bullet-point format. More general or longer-standing policies and structures, including faculty and departmental curriculum committees and the traditional external examiner system, also play important roles in the improvement and assurance of teaching and learning quality.
-
Adoption of a clear statement describing the characteristics that should define the ideal graduate. An example of such a statement is:
"Qualified, competent professionals; proficient communicators, equipped with a wide range of skills; exposed to a range of disciplines and the wider world of scholarship; able and willing to continue to learn."
Teaching and Learning Quality Committees at the institutional and faculty level, ideally including a central Academic Quality Assurance Committee charged with overseeing the whole programme Institutional, faculty, and departmental teaching awards Professional development units specialising in teaching and learning (these units go by different names in the various institutions); clear guidelines for the use of such units should be providedÐfor example, use might be required for new staff and for staff who are experiencing difficulties with their teaching Teaching orientation sessions and assistance for new staff Departmental reviews, by the institution or the relevant faculty, that include teaching and learning processes; these might replace certain institutions' current system of periodic programme revalidations An internal system of explicit and comprehensive teaching and learning quality audits, modelled broadly after the TLQPR. Broadening the visiting examiner system's remit to include teaching and learning quality processes; ensuring that selection procedures for external examiners are rigorous, that the examiners are give proper guidance, and that departments and staff pay close attention to their recommendations
Mandatory student course evaluation questionnaires Student-staff consultative committees Systematic procedures for obtaining curriculum advice and feedback on education performance from employers, former students, and other external stakeholders; ideally, institutions should ensure that departments and programmes obtain such information and provide assistance on how to obtain it Heavier weighting of teaching performance in staff reviews; teaching as a required element of staff review A new post of Educational Development Officer, to work with departments on improving all dimensions of teaching quality Use of one-line budgets based in part on student numbers (and hence on quality and relevance as perceived by students); under some circumstances, quality in relation to cost might inform a departmental decision about whether to pay another department to teach a particular subject/module or to hire someone themselves
|
| 4.3 |
Additional good ideas encountered at the operating unit level include:
organised programmes to develop new teaching materials using video tape, multimedia technology and the internet; peer review of teaching (visiting classes, team teaching, commenting on examination scripts, sharing of teaching techniques); student feedback via direct e-mail links and an electronic 'poll-stand'; adaptation of curriculum to employment patterns of graduates; an active learning environment, self-paced learning systems and 'classstalk' systems for real-time feedback; 'class assessor' schemes for student input/feedback; involvement of undergraduate students in faculty research via 'directed research projects', including the publication of working papers by students; -
open forums and informal tea-gatherings to canvas students' views on individual courses in a relaxed environment; and
a mentor system wherein faculty members provide pastoral care for students via an effective advising system.
|
| 4.4 |
No one institution has adopted all these 'best practices', but some have done better than others. More important than scorekeeping, however, is whether the totality of an institution's policies and practices adds up to a 'culture of quality' for teaching and learning. Such a culture requires more than policies, procedures, and documentation. It requires awareness of, commitment to, and involvement in quality principles and programmes on the part of everyone from institutional head to rank and file academic staff.
|
| 4.5 |
The TLQPR Panel identified numerous areas where quality-process improvement is needed. The most important are summarised below. Each was cited in the TLQPR reports for a number of different institutions.
Variations in quality-process awareness, commitment, and involvement across academic units (faculties, departments, programmes) should be reduced. Examples include differences in the influence of faculty-level quality assurance and improvement processes upon departmental behaviour, the degree to which teaching methods and implementation quality are subjects of active consideration by staff, and differences in the training and performance of teaching assistants. Such variations may be symptoms of insufficiently broad and deep commitment to the institution's quality processes, and their persistence may indicate that the locus of responsibility for assuring and improving quality is unclear. At best they represent opportunities for improving overall quality-process performance by identifying and improving those units that fall below the average on each dimension. Discussions of pedagogical issues (except possibly curriculum design) should be more self-conscious, less informal, more systematic, and less reactive. Focusing attention on pedagogical design, implementation quality, outcomes assessment, and the impact of resource allocation strategies on teaching and learning quality--supplementing the attention traditionally given to curriculum content--will contribute to quality assurance and is essential for continuous improvement. -
The feedback channels for assessing teaching and learning performance should be enhanced. For example,
most course evaluation systems can be improved by clarifying the degree to which their purposes are summative or formative, and providing students with information demonstrating that their inputs make a difference; student-staff consultative committees should be created or improved; external examiner systems should be critically reviewed; and methods of interaction with alumni and employers should be expanded and systematised. Systematic and continuous feedback is essential for both quality assurance and improvement.
Concerns about the balance of teaching and research in staff time utilisation and performance evaluation should be resolved. While research and scholarship contribute to good teaching, they also can take time away from tasksÐincluding quality assurance and improvement tasksÐthat are essential for teaching and learning quality. Academic staff in most institutions perceive that the main rewards are associated with research. Moreover, some institutions appear to have set percentage-of-effort goals for research that are incompatible with long-term teaching and learning quality. Educational development units should become more mainstream, better supported, and more widely used by staff. They should better prioritise and focus their efforts and become more closely integrated with the institution's overall quality strategy. Despite worries about being cast in the role of 'teach police', they should provide diagnostic as well as technical assistance. Institutions, faculties, and departments should promulgate stronger policies about when teachers should call on the units for assistance. Institutional leaders at all levels should ensure that educational quality assurance and improvement programmes are seen as central to the institution's mission, not as bureaucratic appendages imposed by outside authorities. Leaders should ensure that the programmes possess academic vitality and are not simply paper exercises, that they combine formal and informal elements in a simple and understandable way, that they propagate and continuously improve on best practice, and that they empower rather than disempower departments, programme leaders, and teachers.
|
| 4.6 |
At institution after institution, the Panel found itself evaluating whether educational quality assurance and improvement processes have been embedded in core values or whether they are seen simply as activities needed to satisfy bureaucratic requirements. In other words, does the institution exhibit a strong 'culture of quality' and does this extend all the way to the operating unit level -- to the individual academic departments and programmes? The following table describing certain key characteristics of operating units with effective and ineffective quality processes may help explain what we mean by a 'quality culture.' For example, we were more likely to find the quality culture in departments with a strong sense of mission, strong leadership, and a strong and coherent intellectual core. These units feel empowered, and operate energetically at the unit level. In contrast, ineffective departments are likely to have a fuzzy sense of mission, weak leadership, and abdicate responsibility to individual staff rather than taking collective responsibility.
Characteristics of operating units observed to have:
| Effective quality processes |
Ineffective quality processes |
| Strong sense of mission; quality processes are viewed as essential to furthering the mission |
Sense of placidity coupled with a fuzzy view of mission; a low sense of empowerment |
| Staff don't hesitate to enhance quality processes or adapt them to local circumstances |
Staff tend to view quality processes literally, as pro-forma and paper oriented |
| Staff value and act upon student feedback, proactive in efforts to consult with students |
Staff don't seek or act energetically on student feedback, tend to blame the students for being passive |
| Strong and coherent intellectual core |
Weak intellectual core |
| Strong leadership |
Weak leadership |
| Responsibility is assumed at the unit level |
Responsibility is assumed at the course or individual-staff level |
|
Ideas for TLQPR Improvement
| 5.1 |
The continuous improvement principle should apply to the TLQPR process as well as to teaching and learning themselves. As described in the next section, the UGC has provided for external feedback on the process used for the first seven reviews. It is expected that the second-round TLQPR Panel and the UGC will use this feedback to effect improvements. In this section we offer some suggestions based on our close involvement with the first-round reviews. Some of the ideas emerged in mid-stream, but were not implemented because the process had to remain fairly stable during the course of the first round. Others have occurred to us and colleagues on the Panel as we reflected on our experience.
|
| 5.2 |
Our first and most important observation is that the questions put by the Panel to institutional respondents should be somewhat more structured than was possible in the first round. Panel members covered all the needed ground, but now that the TLQPR terrain is better mapped it should be possible to develop a more substantial interview protocol than was possible initially. In addition to ensuring that nothing is overlooked, a more extensive protocol will relieve some of the load on the Panel Chair by making the summarising of findings and report writing easier. We envisage that a standard list of questions would be supplemented by institution-specific questions developed through analysis of the institution's self-assessment document. Panel members should come to the pre-visit meeting with suggestions for institution-specific protocol questions, and the final protocol should be agreed during the meeting.
|
| 5.3 |
The institutions should be given more detailed specifications for their self-assessments and other documentation, and future Panels should insist that the specifications are followed. The self-assessments should address issues of critical importance the TLQPR process. (Some of the issues were discussed in the previous section, and others doubtless will emerge.) Strict bounds should be placed on the quantity of documentation provided. In some instances during the first round, Panel members received documentation on a scale best described in terms of side inches -- volumes that simply could not be assimilated. Because much of the documentation pertained to quality assurance committees and individual-unit processes, it may prove desirable to seek short self-assessments from each committee and unit to be visited in lieu of the more general request for pre-existing documents used this time. It may be necessary to append key documents to the self-assessments to verify that what has been said; however, specific portions of such documentation should be referenced in the self-assessment text. Panel members also might request additional documents in advance of their visit, and a general sampling could be made available for perusal during the visit.
|
| 5.4 |
The visit programme should be adjusted so that less time is spent on large-group presentation-oriented sessions, and more time on small-group interactions with the people actually responsible for delivering and assuring quality. In the first-round programme, the six quarter-day segments (three half-days) were allocated roughly as follows: two for opening plenary sessions, two for small-group interactions, one for the executive session, and one for the plenary exit conference with the institution's leadership. The revised programme might limit the opening sessions to a single segment, allow four segments for the small-group discussions, and use the last segment for the executive session. Better self-assessments would limit the need for panellists to listen passively to presentations (which typically reiterate points made in the documentation). The exit conference may not be needed or its purpose might be better accomplished in other ways. The revised programme would allow the three-member subgroup sessions to be expanded to include the institution-level quality assurance committees. This would make better use of members' time and permit more substantive conversations than is possible in a plenary session. It also would permit more emphasis on faculty-level quality assurance processes, an area which has emerged as critically important.
|
| 5.5 |
To make student participation in the TLQPR more effective, a preparatory seminar might be organised before the main TLQPR visit. The seminar would brief prospective participants on the purposes and methods of the TLQPR, and pose questions from the general protocol for which student input would be particularly helpful. Participants might be asked to consult with fellow students before the TLQPR visit.
|
| 5.6 |
The final report should be organised around the general and institution-specific protocol questions, with less space being taken for general discussion of the TLQPR's goals and methodology. The subgroups would continue to produce written reports (this proved to be one of the more valuable aspects of the first-round process), but the reports should be organised by protocol question rather than entity visited. Each subgroup would produce a single report, citing material from the individual meetings by way of example. The subgroup reports would be integrated into the main Panel report rather than being published as an annex. Thus the main report would include representative 'slices of life' without the length and redundancy characteristic of the first-round report annexes.
|
| 5.7 |
The TLQPR Panel and UGC Quality Sub-Committee should continue and expand their efforts to identify and publish quality-programme best practices. Descriptions of selected best practices might be included as an annex to each institutional report (the body of the report would be mainly summative), with the annexes summarised in an integrative document at the end of the exercise. Development of the integrative report could be facilitated, and the practices themselves promulgated, by an inter-institutional 'best practices' working party.
|
|
|
The Way Forward
| 6.1 |
In anticipation of the completion of the first round of TLQPRs, the UGC and the TLQPR Consultative Committee agreed that a seminar should be held in April 1997 to provide an opportunity for the institutions to share experiences and examples of best practice identified during the course of the Reviews. The seminar duly took place and was attended by TLQPR Panel members, the Consultative Committee and some 150 other colleagues from the institutions. It was also used as an avenue for discussing an exposure draft of this paper and much of the material described in sections 3 - 5 above derives from discussions during the seminar.
|
| 6.2 |
During discussions with the Consultative Committee about the proposed seminar, a further suggestion was raised of undertaking a more formal evaluation, via an outside consultant(s), of the impact of the TLQPRs. Specifically it was suggested that the following questions should be addressed:
What were the original aims of TLQPR? Have these aims changed over the course of the initiative? If so, why? Have these aims been clearly communicated to the UGC-funded higher education institutions? Have the institutions interpreted TLQPR in the same way? To what extent has TLQPR affected institutional teaching and learning processes? Have the TLQPRs resulted in any substantial impact on teaching policy and practice in the various institutions? Are there any outcomes that might be considered as unintended in regard to the stated aims of TLQPR?
|
| 6.3 |
The UGC accepted in principle that such an evaluation should be undertaken, but decided that, in view of the Committee's and the institutions' other activities and commitments, the earliest that it could be initiated would be in late 1998. This timing is also felt to be more appropriate in view of the UGC's expectation that the institutions would themselves undertake a review of progress with the implementation of measures in response to the areas for improvement identified in their TLQPR reports. These reviews are expected to be undertaken so that a report can be made to the UGC on progress about two years after the review visit to each institution.
|
| 6.4 |
For the future, the UGC has agreed that a TLQPR of the eighth and newest UGC-funded institution, the Hong Kong Institute of Education, should be provisionally scheduled for January 1999. Thereafter a second round of TLQPRs of all the UGC-funded institutions would probably be initiated during the 2001 -2004 triennium.
|
|
|
Conclusions
| 7.1 |
We have described Hong Kong's first teaching and learning quality process review for tertiary education. Reviews have now been completed for all seven of institutions for which the UGC was responsible at the time the exercise was undertaken. Five of the seven reports have been published. The TLQPR was designed to focus attention on teaching and learning as the primary mission of Hong Kong's tertiary institutions, to assist institutions in their efforts to improve the quality of teaching and learning, and to enable the UGC and the institutions to discharge their obligation to maintain accountability for the quality of teaching and learning. The evidence to date indicates that the exercise has achieved its purposes.
|
| 7.2 |
The TLQPR has reassured the UGC that teaching and learning processes in the seven institutions are broadly satisfactory. The Panel observed numerous examples of good practice, which have been documented and made available across the tertiary education sector. We also found many areas where improvement in educational quality processes is needed, and these have been brought to the attention of the institutions and the public. The TLQPR process proved strenuous for all concerned, but we believe the outcome will prove to have been worth the effort.
|
| 7.3 |
The basic premise behind the TLQPR--that the efficacy of educational quality processes can be determined through self-study corroborated by interviews at the institutional, faculty, and departmental levels -- has been demonstrated satisfactorily. The ideas for improvement presented herein will make the process even more effective. We know of no better approach for achieving the aforementioned goals without compromising institutional autonomy and disempowering the very people who must deliver, assure, and continuously improve quality. We are convinced that the TLQPR process is worthy of replication, both in Hong Kong and in other countries.
|
|
|
|
|
|
