|
Format
8:15 - 8.45 am Registration
8:45 - 10:30 am (A) - Opening remarks by the UGC Chairman
- Presentations from seven institutions
- Plenary Discussion
Coffee break
11:00 am - 12:00 - Presentation from Professor Bill Massy (Chair, TLQPR Panel) and Mr Nigel French, UGC Secretary-General
- Plenary Discussion
12:00 - 1:00 pm Lunch
1:00 - 2:30 pm (B) - Discussion Groups
- Preparation of reports/recommendations
Coffee break
3:00 - 5:00 pm (C) - Reports from Discussion Groups
- Open Forum
5:00 - 5:30 pm (D) - Seminar Summary by Professor Bill Massy
- Closing Remarks by Professor Kenneth Young, Chair, UGC Quality Sub-Committee.
5. Summary of Opening Remarks
Speakers:
Prof Samuel Chan, The University of Hong Kong (Seminar Chairman)
Prof Kenneth Young, The Chinese University of Hong Kong
Chairman's Opening Introduction
Good morning ladies and gentlemen. Let me begin by welcoming you all to this TLQPR Seminar, especially members of the TLQPR Panel and UGC Quality Sub-Committee, student representatives, and those organizations which did not participate in the TLQPR, namely HKCAA, HKAPA, HKIEd, OLI & VTC.
It has been four years since the Government agreed, in 1993, to a UGC recommendation that there should be periodic process reviews of all UGC-funded institutions. The first round of TLQPR conducted by the UGC in partnership with the institutions has now been completed. In August 1996, the UGC agreed that a TLQPR seminar should be organized after the completion of the first-round reviews. The aims of the seminar are:
to provide an opportunity for the TLQPR Panel and the institutions to share experience and examples of good practices; to enable the UGC and the Panel to obtain feedback on the TLQPR process itself from the institutions; to provide an opportunity for discussion of the initial draft of a summative report on the first round of TLQPRs which the Panel planned to produce.
This seminar is organized by the Consultative Committee in partnership with the UGC. In view of the aims above, I suggest that our discussions and comments should be forward-looking and constructive. Views and conclusions derived from this seminar would be both summative (i.e. to determine the extent to which stated aims have been achieved) and formative (i.e. to recommend possible improvements for future TLQPR exercises).
The first part of the Seminar is Opening Remarks and presentation by institutions' representatives. It is my pleasure to call upon Professor Kenneth Young, Chairman of the UGC Quality Sub-Committee, to deliver his Opening Remarks.
Professor Kenneth Young
Prof Chan, colleagues, ladies & gentlemen, on behalf of the University Grants Committee, I would like to extend my warmest welcome to all of you in attending this TLQPR Seminar.
Mr Antony Leung, Chairman of the UGC has asked me to convey his apologies for not being able to attend this Seminar.
I would like to thank Prof Samuel Chan, who has kindly agreed to chair this Seminar, the Consultative Committee on TLQPR and the Organising Committee, especially Mr Bradford Imrie, for all the efforts put into the planning and organising of this major event.
I would like to thank in particular the student representatives who are participating in the Seminar today. Your contribution and feedback as one of the main user groups are vital towards continuous improvement in the quality assurance systems in our tertiary institutions.
The UGC attaches great importance to assuring the quality of higher education. Let me recap the goals of the TLQPR - to focus attention on teaching and learning as the primary mission of Hong Kong's tertiary institutions; to assist institutions in their efforts to improve the quality of teaching and learning; and to enable the UGC and its funded-institutions to discharge their obligation to maintain accountability for the quality of teaching and learning.
The UGC as well as the Consultative Committee consider that the first round of TLQPRs has achieved these goals.
Let me also stress that the design of the TLQPR is centred around the spirit of collaboration and mutual learning. My colleagues in the Review Panel, especially those from the eight UGC-funded institutions, have pointed out that both the Panel and colleagues have learnt a great deal during the process
We hope that colleagues can make use of this Seminar to share examples of best practice and the experience learnt. In this spirit, the purpose of this seminar is to obtain feedback on the following:
the extent to which the TLQPR exercise has achieved its stated aims the impact of TLQPR on institutions (including unintended outcomes) outcomes in terms of identification of good practices for dissemination improvements to be made for future TLQPR exercises.
It is our belief that everyone involved in these quality assurance systems could continue to improve ourselves and we hope that TLQPR has delivered the right signals to all concerned of the importance of maintaining continuous and self-improving quality as higher education sector as a whole.
Today, I can see the presence of colleagues from other sectors of higher education (OLI, VTC, HKCAA, and APA) and I do hope that they can actively participate in the discussions and share their experience with colleagues.
I wish everyone a fruitful series of discussions today.
6. Institutional Presentations
Institution: City University of Hong Kong
Speaker: Prof Enoch C M Young, Vice President/Institutional Advancement
A Focus on Teaching and Learning
At the very outset of the TLQPR exercise, in his speech to the 1995 "TLQPA" forum, the Chairman of UGC linked the scheme for Review to the "misunderstanding" that had emerged that UGC was "mainly concerned with the development of research and the assessment of research performance". It is in this context that the Reviews were intended to "focus attention on teaching". At City University of Hong Kong, the Quality Assurance Committee, formed in 1993, had already been working hard to maintain an institutional commitment to high quality teaching and learning. Support for this project from UGC was timely and useful. It is important to note, however, that "reviews" unaccompanied by any consequences for funding are unlikely to correct the bias introduced into the system by UGC's current funding arrangements. Under these arrangements, while research performance is the basis for research funding, student numbers, not teaching performance, are the basis for the funding of teaching. If the UGC seriously intends to create an environment in which an appropriate balance between teaching and research effort can be established, it will be necessary to find ways to link funding with the quality, as well as the quantity, of teaching.
Impact of TLQPR
CityU had acquired a good deal of experience with quality assurance systems before the details of the TLQPR were made known. The University had already established a scheme for Internal Quality Audit that drew from the same international experience that inspired the TLQPR. This audit scheme is faculty-based and involves a review of documents, a visit and a written report. Nevertheless the five "themes" set out by the TLQPR Panel, the vocabulary employed, and the questions that accompanied these themes, were very useful to us as we took stock of the systems we had established. As we revise our own scheme for internal audit and refine our own listing of issues for review, we will undoubtedly incorporate much of the structure of enquiry laid out for the TLQPR. A major impact of the Review was that it placed quality issues firmly on the University's agenda. Staff responsible for the development of quality processes were given an opportunity to consolidate systems, while the external deadlines set by TLQPR were undeniably helpful in accelerating developments. In particular, the resources devoted to internal audits, and the seriousness with which this exercise was taken, was increased by the prospect of external scrutiny of our quality processes. The Review visit was a constructive exercise. It provided an opportunity for staff and students to consider in a systematic fashion, at considerable length, and with the help of experienced external assessors, the processes available to help them improve teaching and learning. Feedback indicates that both the Plenary sessions and unit visits were helpful and will have a lasting impact. The University also benefited from the comments made during the final plenary session and expects to learn more from the final report when it is available. The identification of strengths and of "areas for improvement" will provide a valuable guide to the further development of our quality assurance systems.
Review Reports and "Good Practice"
The University has read with interest the published reports on its sister institutions and those institution's own descriptions of their quality process. These documents collectively provide a unique sharing of experience and an opportunity to learn from others. We were, for example, interested in the different ways in which local universities work with external examiners, and in the mechanisms used to bring students into the quality process. We believe that it would be useful for UGC to prepare a summary of the "good practice" that emerged in the course of the Reviews and to distribute this summary widely.
Future of TLQPR
CityU believes that the TLQPR exercise, taken as a whole, was valuable. It has had a beneficial impact on quality processes at the University, and that must translate into improvements in the quality of teaching and learning. The University would therefore encourage UGC to follow up on this cycle of TLQPR with a proposal for a routine system of process reviews. The development of a more permanent scheme will benefit from a thorough assessment of the recently completed exercise and from intensive discussion with the institutions. This discussion will need to take up a number of important issues, including: the guidelines for review; the documentation to be provided; the size and composition of the Panel; the structure and length of the review visit; the format of Reports; and the process for follow-up. The University welcomes this seminar as a first step in opening up these questions. Not withstanding our generally favourable impression of TLQPR, we noted some tendency for review questions and Reports to confuse the assessment of quality with the assessment of quality processes. For example, at some points, the "themes" and accompanying questions implied criteria against which to assess outcomes, not processes. Also, in our view, judgements about the structure of curriculum were out of place in review reports. This is not because universities should be immune from judgements about their curriculum or its delivery, but because the TLQPR process was not designed to enable such judgements to be made. As the UGC and the institutions consider the future of TLQPR the balance between the assessment of quality and the assessment of quality processes will need to be given careful attention. However, it is important to be clear that good design of the next cycle of TLQPR, cannot guarantee that the quality of teaching and learning will get the attention it deserves. It is also necessary to address the bias in funding arrangements noted above. As long as research funds are earned by performance, while teaching funds are given out on the basis of student numbers, there is an overwhelming financial incentive to improve research quality, and no financial incentive to do more than maintain a threshold standard for teaching and learning.
6. Institutional Presentations (cont'd)
Institution: The Chinese University of Hong Kong
Speaker: Prof K C Lam, Chairman, Department of Geography
Reflections on the Teaching and Learning Quality Process Review
The Chinese University always regards the quality of teaching and learning as being fundamental to the overall quality of an institution, irrespective of its particular role and mission. The University is thus committed to the maintenance and continuous improvement of its teaching and learning quality. In this connection, the University welcomes the recent teaching and learning quality process review, not only because it helped us focus attention on the importance of university teaching, but also because it brought forth views and insights from the fresh angle of a third party.
Back in early 1994 when the UGC recommended that a Teaching and Learning Quality Assurance Assessment Exercise be conducted in the form of periodic external reviews, the University already demonstrated its full support, driven by its firm belief that some form of external monitoring of the institution's quality assurance processes was necessary in the interests of public accountability and beneficial for the further development of the teaching and learning quality process. The University was also of the view that the review should be based on a continual dialogue between the UGC and the institutions and the exercise should focus on the processes of quality assurance adopted by the institutions. Therefore the University fully supported the change of the emphasis of the exercise, from an assessment of the quality, to a review of the processes.
The University is mindful that the mere existence of a set of mechanisms and activities is no guarantee for quality. We firmly believe that for quality teaching to prevail, there has to be a quality culture which embraces all aspects of the policies, procedures and practices of the University and its academic departments. The opportunity was taken by the University to examine the appropriateness and adequacy or otherwise of university-level policies and guidelines and to remind all its staff members of the importance of the teaching function. Every faculty and department was called upon to familiarize with the requirements of the exercise and to critically discuss and review its quality assurance processes already in place in Board meetings. To help academic units selected for the TLQPR visit in January 1996, internal audits were conducted in the run-up to the visit.
The University was fully aware that the quality culture does not just happen. It calls for vision, leadership, well-designed processes and diligent follow-up. A Special Task Force on Teaching and Learning Quality was formed in May 1995 with the University Vice-Chancellor as its convenor to help the University prepare for the TLQPR visit, to conduct internal process audits, and to advise on activities related to the enhancement of teaching and learning quality, including the administration of the Teaching Development Grants.
The University believes that the mechanisms for quality assurance in teaching and learning in place have hitherto been enabling the University in pursuing its education mission. However, the University also believes that all its quality assurance processes should be subject to constant review, as they have always been, to ensure their effectiveness in changing times. Further steps are being taken by the University to further enhance the quality of teaching and learning. Having contributed substantially to the preparation of the TLQPR visit, the Special Task Force is now functioning on a permanent basis to continuously review and promote teaching and learning quality at the University. The setting up of the Teaching Development Unit by using part of the Teaching Development Grant is another concrete step taken by the University to improve its existing quality assurance processes for even greater effectiveness.
The University recognizes that for the quality culture to take root, its staff must be conversant with the mechanisms of quality already in place and its academic units must be committed to the continuous improvement of such mechanisms. In this regard, the University is pleased to report that the TLQPR visit has prompted many academic units to rethink about their mission, examine their quality assurance mechanisms and embark on plans for further improvement. Each department is encouraged to examine its own strength and weakness and to formulate an appropriate strategy in the light of its vision and unique nature of the discipline.
The Geography Department, for example, has benefited greatly from the TLQPR visit. Like many other academic units at The Chinese University, the Geography Department had developed, before the TLQPR visit, a variety of strategies, procedures and processes for assuring quality in teaching and learning. The internal process audit, and the subsequent TLQPR review, prompted the Department to engage in a self-appraisal of the system and to formalize good practices, and precipitated an action plan for further improvement.
In preparing for the visit, the Department involved both staff and students to examine its mission and to the review of the TLQ assurance mechanisms already in place. Key issues were discussed in staff retreats and open fora with students and expert advice was sought from the University's Teaching Development Unit. The Department's efforts in fact did not stop after the TLQPR visit. Even before the release of the TLQPR report, the Department drew up a self-improvement action plan the implementation of which has been closely monitored and reviewed in subsequent staff meetings and retreats.
In as much as the goal of the TLQPR is to foster the development of a quality culture in teaching and learning, we believe the review exercise has served the purpose well. The review has quickened the development of a quality culture to which academic departments are obliged to respond.
Again, taking the Geography Department as an example, this quality culture has been enhanced, through the review exercise, by a number of mechanisms, including sensitization of the university staff, interaction among faculty members and with students to facilitate cross-fertilization of ideas, incentives for good practices, and support for creating a quality teaching and learning environment. Details of the approach adopted by the CU Geography Department are:
Sensitization
Collective Individual
- Staff review, Staff response to course evaluation results, Teaching portfolio
Interaction
Workshops/Teaching Cell/Staff Retreat Peer evaluation, Reflective Practitioning Dialogue between Department Chair and Faculty Special Staff-Student Forum
Incentives
Support
In summary, the TLQPR has been a catalyst to the University in its pursuit of excellence in teaching. The initiatives taken by the University administration have been echoed at all levels. Academic units are earnestly looking for ways to improve. Both the University and the Department are committed to review the quality assurance mechanisms periodically.
6. Institutional Presentations (cont'd)
Institution: Hong Kong Baptist University
Speaker: Prof Jerry Barrett, Academic Vice-President
Reflections on the Teaching and Learning Quality Process Review (TLQPR) Exercise
The Extent to Which the TLQPR Exercise Has Achieved its Stated Aims
If the three aims of the TLQPR could be summarised as: "to focus attention on, to assist institutions in their efforts to improve, and to assure all the publics (stake-holders) that teaching and learning are taken seriously and are of good quality, yet continuing to improve", then the exercise seems to have achieved its basic objectives. Just as there is always room for improvement within institutions in these aspects of the most important part of their missions, TLQPR has room for improvement as well. Some observations and suggestions on possible improvements are made under section 4 below. From the perspective of Hong Kong Baptist University, we consider the aims have been achieved rather well. It is always difficult for an institution, to convince persons that it does place teaching and learning in the primary position of its overall mission when so much emphasis is publicised in regard to research and scholarship, but reviews and comments from outside peer reviewers, especially if there are no special links to the institution, can be more persuasive. HKBU has even had some difficulty convincing the UGC of its own intentions in this regard.
The Impact of TLQPR on Hong Kong Baptist University
Hong Kong Baptist University has two commitments that are salient to the discussions about whether the TLQPR exercises have accomplished its stated aims:
it has a long tradition of emphasising good teaching and learning and it is committed to peer review processes to ensure that such good practice does indeed exist.
It could be added that in life, both for individuals and for institutions, whether aims are achieved or not is often left to rather uncritical assessments. As long as practices prove adequate, we rarely call them up for careful examination; doing so primarily when practical routines begin to give problems or are failing. Teaching and learning are far too important for this approach to be our modus operandi.
Having a regular external process such as the TLQPR exercise, provides the potential to achieve both the aim of good teaching and learning and of maintaining peer review monitoring, to which the University is committed. Furthermore, such an exercise enables the University to carefully examine its processes prior to any problems arising that could jeopardise the quality of its teaching and learning. A TLQPR-type exercise enables the University to have a mirror held up to examine the way it goes about attaining/maintaining quality in teaching and learning - a kind of pre-emptive strike against uncritical complacency regarding the teaching mission of the University.
Outcomes in Terms of Identification of Good Practices for Dissemination
The fact that TLQPR Reports are shared between institutions and, in due course, released to the public is helpful in providing information on good practice to all academic institutions. Of course, comments about what is not such good practice can be equally valuable for improvements of all the institutions' processes, not just the one concerning whom the comments are directed.
Improvements to be Made for Future TLQPR Exercises
Suggested changes for future TLQPR exercises:
Greater time for the final feedback session to confirm conclusions drawn by Panel members, partly to ensure their accuracy and partly to explore the wider context in which the issues exist; every effort should be made to understand whether some issues raised were actually outdated; To form a translation committee with a representative from each institution, prior to the first institutional visit, for the purpose of agreeing the Chinese language terms that are to be standardised for use in all the institutional reports; To reconsider whether the "Individual Unit Observations" are to be made public, rather than provided only to the institutions for their benefit or, possibly, in summarised format for release to the public; To improve the format of recording the "Individual Unit Observations" by including paragraph numbering and by using the generic term "unit" rather than sometimes referring to "department", sometimes "course", sometimes "school", or other references; and To avoid as much as possible reporting statements such as: "A faculty member pointed out ..." or "As one Panel member observed ...", since such comments give no evidence whether, on the one hand, the unit concerned agreed with the "faculty member" or, on the other, the Panel agreed with its one member's observation.
In reviewing the commitments to teaching and learning in future, it is hoped that UGC will adhere to the approach that, over time, was adopted for the first round, that is to examine institutional processes within a review format and not to move to "audits" or some other approach. There is some anxiety about the time spent on reviews and having them too frequently (without good cause) or making them more onerous could be counter productive.
These exercises could turn out to be something akin to the measurement of atomic and subatomic particles: you can perform measurements and get a pretty clear idea about energies (momenta) or positions, but not both at the same time (Heisenberg). Indeed external measures of teaching and learning may find the "position" of these activities at some particular moment, but only by disrupting momentum. If TLQPR-type "interventions" are not properly undertaken, the natural momentum of those conscientious academics towards good teaching may be restrained and artificially confined to "the expected acceptable locations".
In conclusion, we welcome peer reviews in which mutual trust and common purpose prevail, but we should guard against excessive intrusions into these highly complex aspects of institutional life - aspects that certainly require more than visits of a day and a half (or even much, much longer) to explore. At the macro level, however, the TLQPR has proved helpful and has assisted the University in attaining its own objectives of teaching and learning excellence. Accordingly, we support continuing efforts to ensure teaching and learning processes are in place that will give institutions a greater likelihood of good performance in this prime mission of higher education in Hong Kong.
6. Institutional Presentations (cont'd)
Institution: The University of Hong Kong
Speaker:Prof Samuel T H Chan, Pro-Vice-Chancellor (Academic)
The Extent to which the TLQPR exercise has achieved its stated aims
The University of Hong Kong (HKU) was the first institution reviewed in the TLQPR exercise. Because of the novelty of a UGC teaching and learning quality review, the impact on the institution was considerable and far reaching, particularly during the time when the UGC and the institutions were exploring the review procedures and final arrangements, and Departments were waiting for the announcement by UGC on the units to be visited. Teaching and learning have always been the primary mission of HKU, and the TLQPR exercise has undoubtedly achieved its goals by providing a clear message from the funding body to academic departments where there were often tensions between research and teaching in the competition for staff time and attention. Because of the UGC Research Assessment Exercise, most if not all departments and faculty members tend to shift their resources and emphasis more to their research and related activities. In HKU, the TLQPR exercise, many departments have refocused their attention on teaching and learning as their primary mission.
Impact of TLQPR on institutions
The observations and comments, both general and specific, and the suggested areas for improvement in the TLQPR Report have a tremendous impact on HKU, not only at the senior management level (which the TLQPR Panel found to be "cognizant of the requisites for quality and to be making diligent efforts to incorporate quality improvement process into the institution's culture") but also, more importantly, at the level of academic units and staff, some of whom still hold traditional views of academic autonomy. As a matter of fact, HKU embarked on a number of quality assurance and improvement measures on teaching and learning a few years prior to the TLQPR exercise. The TLQPR exercise was therefore crucial in providing external input and in boosting the efforts of the senior management and Faculties to further strengthen the strategies and framework of various quality assurance systems. This is in line with one of the key TLQPR quality dimensions by which the institution and its teaching units work to continue improve teaching and learning quality, and assure themselves that the activities are appropriate and well executed. In response to the TLQPR Report, HKU Senate has introduced a number of measures, both enhancing on-going processes and starting new initiatives, to ensure that teaching and learning quality remain a top priority in the University's mission:
Enhancing and improving the long standing QA processes; Establishing clear lines of accountability, and improving self-assessment & reporting system; Emphasizing the importance of quality teaching in the context of the academic culture of the institution; Initiating institutional curriculum reform; Enhancing and broadening students' learning experiences; Establishing systematic evaluation of output quality and "fitness for use"; Promoting and supporting pedagogical and technological initiatives to enhance teaching and learning quality.
Outcome in terms of identification of good practices for dissemination
In an old institution with a long-standing traditional culture of academic autonomy and a highly devolved and decentralized institutional governance system, it was not surprising, and indeed inevitable, to have great disparity among various academic units in their quality culture and processes for teaching and learning. The TLQPR Report, together with the University's response to recommendations in the Report, were widely distributed and discussed by the Teaching Quality Committees, and at Faculty and Departmental levels. They were also placed on the World-Wide-Web for consultation by all members of the University and the general public. Units with good practices as highlighted in the "Unit-level Observations" section of the report were particularly referred to, so as to stimulate and assist faculties and departments, by following the examples of these units to develop their own collective teaching culture and effective quality programmes.
Improvements to be made for future TLQPR exercise
The assurance of teaching and learning quality is rightly a joint concern and responsibility of the UGC and the higher education institutions (HEIs), and it was certainly appropriate for the UGC to involve the HEIs as partners and conducted the TLQPR exercise as a joint effort. This arrangement, in our opinion, should continue regardless of whether the future exercise(s) on quality is a process review or quality assessment. The declared UGC assumptions, dimensions and methodology for the TLQPR exercise have interestingly led to "standardized" documentation and presentation produced by some units of the institutions visited by the Panel at the later phase of the exercise. Given that the missions, historical characteristics and roles of the local HEIs are not exactly identical, due considerations should be given to devise some mechanisms or arrangements in future to avoid "conformity" behaviour. The future framework for the Teaching and Learning Quality Process Reviews should also call for a more mission-related process review (or assessment if such process is chosen)with special reference to the characteristics of each institution.
Finally, there is no doubt that the process of teaching and learning should continue to be determined entirely by the HEIs themselves and the TLQPR exercise, which aims at assisting the HEIs in their effort to improve the quality of teaching and learning, should remain a joint effort of the UGC and HEIs. The exercise, however, may better serve its goals if some clear statements are to be made by the UGC in regard to whether , and how, the results of the future TLQPR exercise may inform funding.
6. Institutional Presentations (cont'd)
Institution:Hong Kong Baptist University
Speaker: Prof Jerry Barrett, Academic Vice-President
Reflections on the Teaching and Learning Quality Process Review (TLQPR) Exercise
The Extent to Which the TLQPR Exercise Has Achieved its Stated Aims
If the three aims of the TLQPR could be summarised as: "to focus attention on, to assist institutions in their efforts to improve, and to assure all the publics (stake-holders) that teaching and learning are taken seriously and are of good quality, yet continuing to improve", then the exercise seems to have achieved its basic objectives. Just as there is always room for improvement within institutions in these aspects of the most important part of their missions, TLQPR has room for improvement as well. Some observations and suggestions on possible improvements are made under section 4 below. From the perspective of Hong Kong Baptist University, we consider the aims have been achieved rather well. It is always difficult for an institution, to convince persons that it does place teaching and learning in the primary position of its overall mission when so much emphasis is publicised in regard to research and scholarship, but reviews and comments from outside peer reviewers, especially if there are no special links to the institution, can be more persuasive. HKBU has even had some difficulty convincing the UGC of its own intentions in this regard.
The Impact of TLQPR on Hong Kong Baptist University
Hong Kong Baptist University has two commitments that are salient to the discussions about whether the TLQPR exercises have accomplished its stated aims:
it has a long tradition of emphasising good teaching and learning and it is committed to peer review processes to ensure that such good practice does indeed exist.
It could be added that in life, both for individuals and for institutions, whether aims are achieved or not is often left to rather uncritical assessments. As long as practices prove adequate, we rarely call them up for careful examination; doing so primarily when practical routines begin to give problems or are failing. Teaching and learning are far too important for this approach to be our modus operandi.
Having a regular external process such as the TLQPR exercise, provides the potential to achieve both the aim of good teaching and learning and of maintaining peer review monitoring, to which the University is committed. Furthermore, such an exercise enables the University to carefully examine its processes prior to any problems arising that could jeopardise the quality of its teaching and learning. A TLQPR-type exercise enables the University to have a mirror held up to examine the way it goes about attaining/maintaining quality in teaching and learning - a kind of pre-emptive strike against uncritical complacency regarding the teaching mission of the University.
Outcomes in Terms of Identification of Good Practices for Dissemination
The fact that TLQPR Reports are shared between institutions and, in due course, released to the public is helpful in providing information on good practice to all academic institutions. Of course, comments about what is not such good practice can be equally valuable for improvements of all the institutions' processes, not just the one concerning whom the comments are directed.
Improvements to be Made for Future TLQPR Exercises
Suggested changes for future TLQPR exercises:
Greater time for the final feedback session to confirm conclusions drawn by Panel members, partly to ensure their accuracy and partly to explore the wider context in which the issues exist; every effort should be made to understand whether some issues raised were actually outdated; To form a translation committee with a representative from each institution, prior to the first institutional visit, for the purpose of agreeing the Chinese language terms that are to be standardised for use in all the institutional reports; To reconsider whether the "Individual Unit Observations" are to be made public, rather than provided only to the institutions for their benefit or, possibly, in summarised format for release to the public; To improve the format of recording the "Individual Unit Observations" by including paragraph numbering and by using the generic term "unit" rather than sometimes referring to "department", sometimes "course", sometimes "school", or other references; and To avoid as much as possible reporting statements such as: "A faculty member pointed out ..." or "As one Panel member observed ...", since such comments give no evidence whether, on the one hand, the unit concerned agreed with the "faculty member" or, on the other, the Panel agreed with its one member's observation.
In reviewing the commitments to teaching and learning in future, it is hoped that UGC will adhere to the approach that, over time, was adopted for the first round, that is to examine institutional processes within a review format and not to move to "audits" or some other approach. There is some anxiety about the time spent on reviews and having them too frequently (without good cause) or making them more onerous could be counter productive.
These exercises could turn out to be something akin to the measurement of atomic and subatomic particles: you can perform measurements and get a pretty clear idea about energies (momenta) or positions, but not both at the same time (Heisenberg). Indeed external measures of teaching and learning may find the "position" of these activities at some particular moment, but only by disrupting momentum. If TLQPR-type "interventions" are not properly undertaken, the natural momentum of those conscientious academics towards good teaching may be restrained and artificially confined to "the expected acceptable locations".
In conclusion, we welcome peer reviews in which mutual trust and common purpose prevail, but we should guard against excessive intrusions into these highly complex aspects of institutional life - aspects that certainly require more than visits of a day and a half (or even much, much longer) to explore. At the macro level, however, the TLQPR has proved helpful and has assisted the University in attaining its own objectives of teaching and learning excellence. Accordingly, we support continuing efforts to ensure teaching and learning processes are in place that will give institutions a greater likelihood of good performance in this prime mission of higher education in Hong Kong.
6. Institutional Presentations (cont'd)
Institution:The Hong Kong University of Science &Technology
Speaker:Prof Peter Dobson, Associate Vice President for Academic Affairs
Achieving the Stated Aims
Focus attention on teaching and learning-
There is ample evidence that the TLQPR exercise was successful in highlighting the importance of teaching and learning. Before and after the TLQPR visits, the level and intensity of discussion of educational issues have been accelerated. Much more thought has gone into the processes by which we encourage and monitor quality.
Assist institutions to improve teaching and learning-
It is probably too soon to tell whether this will be achieved. Most institutions will "salute the flag" and say the exercise has been helpful, but it remains to be seen whether all the activity stimulated will result in real change and improvement. The deluge of paper that nearly drowned those of us who served on the Panel could be symptomatic of an overly bureaucratic response. But I am optimistic that, when the flurry of self-promotion dies away, there will be a real core of significant change.
Maintain accountability for the quality of teaching and learning -
The UGC and the institutions can, almost by the act of saying it is so, make us accountable by declaring the TLQPR process the official instrument for accountability. This will work partly because external perceptions will most likely not take notice of the difference between a review of quality assurance processes and a review of teaching and learning quality.
On the other hand, the introduction of TLQPR, in the short term, may have had almost the opposite effect of that intended. In an ideal world, the public would view the exercise as evidence that the UGC and the institutions are committed to quality.. But it seems to me that many reports on the subject start from the viewpoint that we would not be doing this at all if there were not something wrong in the first place. So, while there may be a perception that the exercise is a good thing since it is aimed at improvement, the perception may also be that improvement is needed because teaching and learning quality is currently very low. We can only hope that in the long term, as TLQPR or an improved version of it becomes a regular feature of institutional life, the negative connotations will be gradually fade.
Improving the Process
More people-
The thing I value most from involvement in this exercise is the experience of seeing first hand what is happening at the other institutions in Hong Kong. I feel it would be good to involve more people in future versions of TLQPR, so this experience can be shared. This could be done without enlarging the Panel, simply by having different institutional representatives for different visits. Now that we have been through the process once, it is probably sufficient for continuity that the core group from UGC be the same for all reviews. Instead of relying on its representative on the Consultative Committee, with perhaps an alternate, each institution could limit its representatives to at most two reviews. Having more people at each institution really familiar with how the review process works, and how other institutions are responding to it, would be one of the best ways to share "good practice".
More time-
There is little doubt that the most valuable part of the reviews has been the visits to the individual departments and other units. At present, only one-third of a day-and-a-half visit is devoted to this. Of the remaining two-thirds, perhaps half is used to talk to senior representatives, which has also been important and informative. The other third is discussion among the Panel members, and is also hard to eliminate. We should consider adding another half-day, to allow for a full two day visit to each campus, in order to switch the balance towards more on-site on-the-ground review of what's really going on.
Review quality?
I am troubled by the fact that in the TLQPR exercise, we consciously try not to look at delivered quality, but we find it almost impossible not to draw conclusions about it. Reviewing processes alone is a very sterile exercise. Even when we try to limit our discussion to processes, quality itself creeps in. An example is the claim in a recent draft report along the lines that the processes in place give us confidence that they will promote teaching and learning quality. In fact, such a statement has no credible empirical basis. This whole area of quality assurance is so new, as our Panel Chairman often reminds us, that we really do not know what processes will work. I do not mean we can't recognize processes that look as if they should work, and it is probably appropriate to praise those who have adopted them. But if in the future we continue to avoid looking at real quality as it is found on the ground, we will have no way to know if the conclusions we have so confidently stated are in any way justified.
I do not propose that we replace TLQPR by quality assessment. The arguments against that remain as sound as ever. However, I would like to propose that if we expand the visits by a half day, as proposed above, the extra time be spent observing real teaching and learning in real situations, rather than simply visiting more departments and talking about processes. With 18 Panel members, each sitting in on two or three learning opportunities, we could at least get a flavor of how all these processes translate into action. This would not be a quality review, of course. It lacks the depth and focus for that purpose. However, to use a phrase taught us by the Chairman of the UGC, observing the teaching and learning going on could at least "inform" our judgments about the efficacy of the processes.
6. Institutional Presentations (cont'd)
Institution:Lingnan College
Speaker:Prof Barry Bannister, Associate Vice-President (Academic)
Introduction
Lingnan College welcomed the opportunity offered by the visit of the UGC's TLQPR Panel in September 1996 to undertake a systematic self-evaluation of the College's teaching and learning quality processes. We are pleased that the Panel was able to express "considerable optimism that the processes in place were sufficient to ensure the desired quality of teaching and learning at Lingnan."
The Panel's positive feedback confirmed the importance of Lingnan's two key elements of collegiality and close student-staff relationships. In its advice to the College, the Panel urged us not to compromise on these impressive aspects of our cultural identity during the period of rapid change and development which is currently taking place.
Comments in relation to the purposes of TLQPR
Extent to which TLQPR aims were achieved:
The TLQPR assisted Lingnan to further clarify its role as an evolving liberal arts university with an emphasis on teaching excellence. Within this context it also focused our attention upon the need for an appropriate balance between teaching and research while maintaining collegiality and high quality staff-student relationships. The Review also offered us the opportunity to check the comprehensiveness of our learning and teaching quality processes, as well as reviewing feedback systems and making enhancements if necessary. We found that the initial stage which involved the preparation of a reflective self-assessment was useful not only in articulating shared beliefs and practices, but also in providing the stimulus for intensive developmental work with staff and students at the departmental level. This preparatory work, combined with the corporate effort required during the Panel's visit and the invitation to respond to the TLQPR Report, was a beneficial experience which focused the institution as a whole on the primary aims of a university education. As such, both the UGC and Lingnan were able to highlight their joint accountability for the quality of teaching and learning.
Impact of TLQPR on institutions
Staff Induction & Development: In recent times the College has significantly improved its staff profile by a carefully designed programme of staff recruitment including the appointment of a number of Chair Professors. In order to ensure that these staff are fully assimilated into the Lingnan culture of learner-centred teaching, the Educational Development Office and the Lingnan Foundation Visiting Teaching Scholars have initiated an outreach programme to the academic departments.
The expected outcome is an integrated schedule of staff development activities which responds to the teaching priorities of the Departments and Faculties. This, combined with a system of mentoring, is a more productive way of ensuring fruitful participation by new and existing staff, than the introduction of a mandatory staff induction programme. Tutors and newly-recruited staff who have not previously taught, are required to participate in such an induction programme.
Review of Assessment Practices: The Panel suggested that we might consider a further refinement of our processes with respect to External Examiners. External Examiners play a vital role in the present system in Hong Kong in helping to maintain academic standards at a level comparable to those in other tertiary institutions locally and internationally. Nevertheless, following the TLQPR exercise, a review of the entire assessment system is underway and will incorporate a critical appraisal of the external examiner system as practised in the College.
External Examiners are only one of many avenues for obtaining expert external academic advice of use in our overall quality assurance system. In the wider context, the issue of incorporating appropriate and timely external input and review into the academic process is a major consideration in our examination of mechanisms to replace the role of HKCAA in our quality assurance processes after we have achieved self-accrediting status.
Increased Student Focus: Lingnan students were commended by the Panel for their support of the College's distinctive mission and for their understanding of the value and purpose of the General Education component. Following the UGC Panel's visit the College has further emphasised the provision of quality teaching and learning support services, and clearly, student feedback is a vital element of this process. Performance pledges are regularly revised in the light of feedback received, with the aim of providing the most effective and efficient services consistent with resource constraints.
Formal and Informal Quality Processes We noted the Panel's comment on the potential overcomplexity of our committee system which results from a layering of formal processes upon the informal processes already in existence at Faculty and Departmental level. In this connection, two of our degree programmes will be revalidated in May 1997 and a comprehensive Internal Quality Audit is scheduled for September. By that time, we are confident that we shall have integrated the formal and informal processes into a streamlined system which achieves the desired balance between regulation and empowerment; between rigour and wide participation.
Outcomes in terms of identification of 'good practice' for dissemination: One of the purposes of Lingnan's seminar series, Pathways to Excellence in Higher Education has been to disseminate knowledge concerning best practice in teaching and learning. The T & T project housed at Lingnan has also been an important means of dissemination. Apart from these initiatives, there have not been outcomes explicitly related to the TLQPR exercise which have served to disseminate good practice. Four examples of good practice at Lingnan have been cited either by the TLQPR Panel or by others. These are:
Learning Support Network Qualities of the Lingnan student Information Literacy Taskforce Exit tests in English and Putonghua
Improvements to be made for future TLQPR exercises: There have been informal comments made about the TLQPR Panel focusing on matters other than 'process', or of incorporating minority perceptions of institutions into the final reports. However, in relation to possible future TLQPR exercises, it may be appropriate to consider the following two points in more detail:
The teaching and research interface/synergy Statements in relation to possible funding implications of TLQPR
Overall Assessment
In conclusion, we found the various insights provided by the TLQPR Panel to be very useful in our preparations for Institutional Review in January 1998. At that time we will be able to demonstrate that our quality improvement and assurance processes have matured and also that those elements of Lingnan's culture which underpin our mission to strive for excellence in education as a self accrediting institution, have been preserved and enhanced. The timely assistance of the UGC in this process is warmly acknowledged by the whole Lingnan community.
6. Institutional Presentations (cont'd)
Institution:The Hong Kong Polytechnic University
Speaker:Prof Joshua Wong, Vice President (Quality Assurance)
The presentation consist of two parts:
(A) the results of a survey, and
(B) a perspective from VP(QA), Vice-President(Quality Assurance).
(A) Survey Results
About two months after the TLQPR visit to PolyU, a survey was conducted of staff members to gather their opinion of the usefulness of the TLQPR exercise. The response rate to date is 46% with 135 responses from visited units, and 98 from units not visited. In general, staff members from the visited units are more positive about the exercise. The following are the key results.
On the question of how well did the TLQPR exercise achieve its aims, and on a scale of 1 (not at all well) to 5 (very well), the scores are:
| |
Visited |
Not visited |
| |
Mean |
S.D. |
Mean |
S.D |
| (a) Focusing attention on teaching and learning is the primary mission of PolyU |
3.21 |
1.14 |
3.09 |
1.02 |
| (b) Leading to actual improvements on teaching and learning |
2.52 |
1.04 |
2.33 |
0.95 |
| (c) Leading to planned improvements on teaching and learning |
2.95 |
1.10 |
2.64 |
1.07 |
|
