Contents

	1. Appreciation
	2. Organising Committee
	3. Purpose of the Seminar
	4. Seminar Programme
	5. Summary of Opening Remarks
	6. Institutional Presentations
		City University of Hong Kong
		The Chinese University of Hong Kong
		Hong Kong Baptist University
		The University of Hong Kong
		The Hong Kong University of Science & Technology
		Lingnan College
		The Hong Kong Polytechnic University
	7. Massy/French Paper
	8. Group Discussion Reports
	9. Seminar Summary - Professor William F Massy
	10. Closing Remarks
		Professor Kenneth Young
		Professor Samuel Chan
	Appendix 1 Participation According to Organisation
	Appendix 2 Extracts from UGC Report

8. Group Discussion Reports

Group 1 - Students (9 undergraduates; 2 postgraduates)

Presented by Jerome HON Chung-chak, President, Lingnan College Students' Union

(a) The extent to which the TLQPR exercise has achieved its stated aims.

Theoretically, the students think that the objectives set in the TLQPR Plan have been achieved. This conclusion is subject to certain assumptions:

1. perfect communication within the University, especially between the College and students;
2. the report shows a true picture of the institution; and,
3. student feedback was taken into consideration seriously with adequate time of investigation from the root population of the students.

(b) Impact of TLQPR on institutions (including unintended outcomes).

All the students agreed that the TLQPR had a positive impact in helping the universities to improve the quality assurance mechanism with some reservation on those long term suggestions. But not all the people in the University will know about it, especially the students. Information usually was restricted to the College Management/ Academics; in the worst case, no information was brought to the students, unless you are the lucky one who had been chosen to be the representative and face the Panel. The postgraduates also pointed out that their student status was not always clear since they also had teaching responsibilities. They found it quite embarrassing when they were selected as students and did not think the same as the College. It was also raised by the students that the exercises had been a "show" that had affected the daily operation a lot. We would want the UGC to come in a more silent way.

(c) Outcomes in terms of identification of good practices for dissemination.

The most important point was that the TLQPR exercise gave a chance for each of the universities to have a look at quality assurance on teaching and learning. It also pointed out the important role that the general public assigned to universities. The exercise also gave a chance for the student representative to know more about the University. And it has been a general feeling to students that the exercise in a smaller institution tends to have a higher benefit. Most of all, students would like to stress again that dissemination of information about these exercises (including those different reviews by the UGC) should make it essential for the institutions to let the students know about the practices of the TLQPR.

(d) Improvements to be made for future TLQPR exercises.

The follow-up procedures have played an important role in determining the effectiveness of the exercise that should be stated clearly and noted to universities. The duration of the whole exercise (from the first to the last) should be shortened. Contributions from students, both in number of participants and time slots should be extended. Maybe an open forum would be a possible way to improve student participation.

The question on whether the TLQPR should link with funding purposes was discussed. No agreement was reached. Publicity work, especially to the general public, needs a lot of improvement, both at the start of the exercise and at the time that the reports are published. The notice of review to each institution is also recommended to be known to students at the earliest possible time so that student representatives can have more discussion with the root population before the Panel visit. Before the end, the student group suggested that the exercise should be done once per year per institution, unless there are some factors that are not manageable. Three years will be the longest acceptable period for a University to undergo this exercise.

8. Group Discussion Reports (cont'd)

Group 2 - Senior Management

Presented by Mr Jonathon Marsh, Asst Professor, The University of Hong Kong

It was agreed at the outset to divide the group into four subgroups with each of the subgroups being asked to discuss and respond to one of the four seminar purposes. Each group met for 20 minutes and then reported their views. These were discussed by the larger group and feedback comments were agreed upon for the plenary session. These are described below.

Purpose A - the extent to which the TLQPR exercise has achieved its stated aims.

Aim 1) To focus attention on teaching and learning as the primary mission of Hong Kong Tertiary Institutions;

While the group agreed that the TLQPR exercise had helped to focus attention on teaching and learning we did not think that it had established a sense of teaching as the "primary mission" of the institution. Neither did we think that it was appropriate to do so. It was agreed that teaching had been undervalued in the past in relation to research and the exercise had been instrumental in helping to redress the balance.

Aim 2) to assist institutions in their efforts to improve the quality of teaching and learning;

We agreed that on the whole this aim had been achieved. There was a noticeable heightening of awareness among staff, not only of the value of teaching but also of the complexity of the issues involved in ensuring the provision of good quality teaching. The group felt that the exercise had had a strong catalytic effect in providing new momentum and new insight to existing practice. However the group thought that the effect was restricted to improvements in quality assessment strategies and not directly on teaching itself.

Aim 3) to enable the UGC and the institutions to discharge their obligation to maintain accountability for the quality of teaching and learning.

The group felt that, while this aim had been addressed in some measure, it could not be said to be true that a single exercise such as the TLQPR was sufficient to fully discharge an ongoing obligation.

Purpose B - the impact of TLQPR on institutions (including unintended outcomes)

The group felt that there had been overall a positive impact on the teaching culture of the institution. This was reflected by an increase in dialogue on matters pertaining to teaching and a greater willingness to exchange ideas and open up practice to external observation.

There appeared to be an increased concern for improved recognition for good teaching as reflected by the implementation in most institutions of a variety of reward mechanisms (teaching awards, teaching fellowships, teaching development grants, etc.). However there was still a long way to go before teaching could be said to carry appropriate weight in personnel decisions (promotion, substantiation, renewal, etc.).

The exercise was seen to have had considerable impact with respect to accelerating existing initiatives.

There was a considerable increase in public awareness and consequently in the general perception of the need for accountability with respect to teaching.

Purpose C - Outcomes in terms of identification of good practices for dissemination

As a result of the TLQPR, a great deal of work had been done in all institutions to improve both the understanding and the practice of student evaluation of courses and teaching. To a lesser degree other feedback mechanisms and sources of data were also beginning to be explored.

There was an increased understanding and recognition of the complexity of teaching and teaching related duties. Consequently greater emphasis was being placed on the need for training and the need to appropriately recognize teaching credentials.

Some basis had been established upon which to develop performance review systems which were able to appropriately judge and respond to substandard performance.

Purpose D - improvements to be made for future TLQPR exercises

Focus should be directed explicitly on "quality" mechanisms and processes and not directly on teaching quality.

Greater effort should be made to ensure that the evidence provided established the "effectiveness" of the quality processes under review and not simply their existence.

The representativeness, reliability and validity of the documentation needs to be carefully established. Consequently clear guidelines need to be sent out well in advance of the exercise, especially if the results are to be linked to funding.

The review panels should be expanded to include members from within the institution under review in order to ensure that the institution is able to more directly benefit from the deliberation process itself and not solely through the reporting process. All three of the assessment exercises (research, teaching, administration) should be closely coordinated in order to convey a consistent and coherent message.

Personal Comment

While I found the seminar useful in gaining perspective on the other institutions responses to the TLQPR, I did not find it helpful in clarifying directions for future development. As the day progressed it became more and more apparent to me that the issue of "representative" data is critical to future success. I feel more strongly than ever that the review process itself needs to be evaluated in order to ensure that future exercises carry a legitimacy which goes beyond a simple quantification of procedures.

8. Group Discussion Reports (cont'd)

Group 3 - Senior Management

Presented by Mr Ken Stafford, Assoc Director, City University of Hong Kong

Discussion Group 3 initially split into two sections to allow more individual contribution to the topics to come forward. The following points represent the group chairperson's attempt to capture the whole group's comments on the four issues. Obviously, because of the nature of the exercise, there will be items 'missed' and others which have been 'caught'. All observations reflect the interpretation of the recorder and consequently should be interpreted as being general.

The group was able to consider the four issues identified, however, most attention was given to the third and fourth.

To what extent has the TLQPR exercise achieved its stated aims?

• As to the first of the stated purposes of TLQPR, the group agreed that the TLQPR exercise had focused attention on teaching and learning but there was lively discussion as to whether teaching and learning should be seen as the primary purpose of higher education institutions. While the group expressed support for teaching/learning being seen as important, some participants were concerned about the term 'primary' in that it tended to create a competitive rather than collaborative relationship between the two most discussed purposes, i.e., teaching/learning and research.

• There was discussion as to the extent to which the second stated purpose, i.e., to assist institutions improve the quality of teaching and learning, had been achieved by the TLQPR exercise. The group noted two points; (i) it is too early to judge this with any degree of certainty (a clearer understanding will emerge following the formal evaluation of the exercise to be commenced in 18 months time) and (ii) there appeared to be variation in the experience of different institutions. In regard to this feature the members of the group noted that in their experience where the exercise was undertaken in a constructive sincere fashion there were indications of positive outcomes. The implication given was that the opposite also applied.

• The group was less committed to a position as to the extent to which the TLQPR exercise has supported the UGC and the institutions maintain accountability for teaching and learning. Even though a clear position did not come from the group on this issue there was some agreement that it was probably too early to judge and that again a clearer indication would come from the formal evaluation.

What has been the impact of the TLQPR exercise on the institutions?

In responding to this question the group was able to identify a number of specific impacts - some positive, some negative. The most significant of these are reported below.

Positive

• While it was inconsistent across all Hong Kong's higher education institutions, the TLQPR exercise has re-emphasised that sponsoring learning was a (the??) major institutional undertaking which should be recognised, valued and rewarded. The extent to which any reward is actual rather than rhetoric will be tested over time.

• Similarly, the TLQPR exercise has highlighted the contribution of individuals who had regarded teaching as an aspect of scholarship prior to it becoming "fashionable" as a result of TLQPR. This observation is closely connected with the previous one and again the degree to which this contribution will be rewarded will be shown over time.

• The contribution/role of academic/teaching support staff has been recognised. All UGC-funded higher education institutions now have teaching development/support facilities. The range of services offered varies as does the administrative framework for such facilities, however, it is recognised that the support offered is important to enhancement of teaching and learning.

• Overall, the group believed that having established a link between the TLQPR exercise and funding, institutions were taking the exercise seriously. This is not to say that the group agreed unanimously that the results of the exercise should be linked to funding - strong opinions were expressed that this should not be the case. Supporters of this view pointed out that if enhancing learning was the major goal of institutions then linking it to funding seemed superficial. Some group members argued that unless there was a funding link institutions would not take the exercise seriously. (Personal observation: It would be interesting to ask if enhancing learning is not the main purpose of publicly-funded higher education institutions, then what is?)

Negative

• The group was concerned that the TLQPR exercise had left a sense of inconsistency and confusion. It appears that messages sent and received are at times contrasting and confusing. Examples of this include 'the funding link' dependent on performance during the TLQPR exercise and confusion as to how 'process' can be 'reviewed' and then inform funding without some sense of measurable outcomes; the purpose and focus of TLQPR had been misinterpreted in some instances, etc.

• Some participants in the group indicated that they were concerned that particular institutions were responding to the TLQPR exercise in an 'administrative' way. This had led to the proliferation of committees and paperwork with their resultant additional time requirement. As well, such an attitude had resulted in some academics believing that the institution were simply 'putting on a show'. Obviously the extent to which such a fear is grounded will be determined by future actions within the institutions.

Outcomes in terms of good practices for dissemination

The group interpreted this task quite literally and recorded experiences from the various institutions which were thought to be of value and which should be recorded and disseminated for all institutions. As a broad statement, the group indicated that the various teaching support/development centres in the institutions should adopt (if they do not already do so) a clearinghouse function to facilitate dissemination.

• Initially, the group observed that in general the process of documentation was worthwhile and that, specifically, the critical/reflective statements generated by the TLQPR exercise were extremely useful. The actual process of generating these should be recorded and disseminated to facilitate future reflection.

• The group agreed that systems addressing the hiring, development, and promotion of staff with a view to enhancing teaching and learning played a major part in establishing the quality culture of the institutions. Those practices identified during TLQPR visits to various institutions as exemplary in these respects should be disseminated to the widest audience.

• Similarly, it was noted that much useful information affecting the evaluation of teaching and the assessment of student learning had been identified during the TLQPR visits to different institutions. It was agreed that exemplary systems/practices in these areas should be disseminated. (Personal observation: these are in fact focuses for sub-projects in the UGC-funded Evaluating Student Experience Project).

• It was observed that as a result of the TLQPR exercise there was increased activity in creating various forums at which issues focusing on teaching and learning could be addressed. The group agreed that practices adopted by various institutions to promote such forums should be disseminated.

Improvements to be made for future TLQPR exercises

The group recognised that the formal evaluation of the whole initiative will certainly bring forward considerable formative feedback which will advise decisions aimed at improving future TLQPR visits. The observations expressed below represent concrete suggestions based on initial impressions which the group believed would improve the TLQPR visits and provide the panel members with what might be labeled as 'grassroots impressions'.

• Recognising the constraints of man-power, the group believed that the size of sub-panels involved in the actual visit should be varied (contain local and overseas members with general and specific expertise in the particular 'unit' being visited) and larger (4-5 members was suggested). The group believed that even more time should be spent with specific units at the expense of more general institutional groups.

These two suggestions were offered as ways in which the sub-panels could gather greater, more accurate insights into systems and processes in action.

• There was widespread concern that there continued to be different and/or inaccurate interpretations of important aspects of TLQPR. This suggested that dialogue between the TLQPR panel and the institutions (and perhaps among panel members and the UGC) was not satisfactory. It was stressed that a 'clear, consistent message' should come to all institutions and consequently, a strong recommendation is that steps be taken to further strengthen such dialogue.

• With reference to the Panel's reports provided to institutions after TLQPR visits, the group felt that the practice would be more productive if the criteria used for the findings were made transparent. Some group participants indicated that such transparency was lacking in their experience.

• There was agreement that consistent messages as to the purpose and significance of the TLQPR exercise should be sent to all institutions. This is not to argue that all institutions are to be viewed identically since clearly they operate within different frameworks and have greater or lesser experience with certain issues. As well, particular institutions will have their own priorities to which the Panel should be sensitive.

• Following on from the previous point, the group indicated that the TLQPR process would be improved if the Panel displayed more flexibility when considering the individual institutions by taking into account the institution's particular circumstances and context. It seems that a preconceived set of expectations/criteria may adversely affect the report of individual institutions if these where not held to the same extent because of contextual factors.

• Finally, the group suggested that documentation (while considered generally useful) would be improved by better and more streamlined guidelines. There was a feeling among group members that in some instances volume rather than usefulness was the factor most affecting institution's documentation.

Discussion was lively and far-ranging. At times we strayed from the purposes indicated by the organisers. However, the points noted above indicate the recorder's interpretation of the main issues discussed.

8. Group Discussion Reports (cont'd)

Group 4 - Senior Management

Presented by Mrs Pauline Mah, Senior Asst Academic Registrar, Hong Kong

Baptist University

The group comprised over 20 senior members from the UGC funded tertiary institutions, two UGC members and a representative from VTC and APA. The discussion centred on the purpose of the seminar, which is to reflect and obtain feedback on the extent the TLQPR exercise has achieved its stated aims; the impact of the exercise on the institutions; the outcomes in terms of identification of good practices for dissemination and improvements for future TLQPR exercises.

a) Extent that the TLQPR exercise has achieved its stated aims.

One member shared his view that, to a certain extent, the exercise has focused attention on teaching and learning as evidenced by the increased number of applications from staff in his department for teaching development grants to enhance teaching. There was a consensus that the exercise has heightened awareness on teaching and learning but the extent and effect of this will not be visible for some time. It would therefore be prudent to sustain the momentum that has been created. This could be in the form of an annual conference to focus attention on teaching, where academics can discuss and share local problems in teaching (separate from the international Improving University Teaching Conference) and to enable dissemination of good practices in teaching. The third stated aim, which is to enable the UGC and the institutions to address accountability in teaching and learning to the public, was viewed as successfully accomplished.

b) Impact of TLQPR on institutions (including unintended outcomes).

A query was raised on the statement that, teaching and learning is the "primary mission" of Hong Kong's tertiary institutions. The consensus was that a balance between teaching and research would be a more accurate description of the mission of the institutions. The TLQPR exercise re-affirmed the importance of quality in teaching and learning.

At the same time, the research assessment exercises have greatly emphasised the importance of research in universities. To academics, the two exercises appear to be imparting confused messages on which is of primary importance. Thus, the idea of the UGC implementing different "waves" of review exercises and creating confusion on what is expected of staff in the institutions, was perceived to have taken place. To alleviate this confusion, a suggested proposal was for the UGC to consider a review of teaching, research and services on equal footing in one exercise.

One of the group members mentioned that a number of institutions are playing an active role in the professional development of students, yet, it appears that this role has been ignored by the UGC in its review exercises.

Some members felt that the exercise was to a certain extent considered disruptive to the normal activities of the institutions as very limited time was made available for institutions and units to decide on what materials to include to substantiate their statements and to locate the materials for despatch to the UGC. This came about because the selected units to be reviewed were not informed of their selection earlier and were given limited time to assemble their materials.

c) Outcomes in terms of identification of good practices for dissemination.

A variety of good practices exists in different universities and a suggestion was made for the UGC to disseminate these practices and the institutions to help departments to improve. The UGC could reinforce the practice of dissemination while the institutions take the initiative to do the job. Request for dissemination of the results of the present TLQPR exercise was also made.

The need to develop a continuous quality culture in the area of teaching and learning was encouraged.

The UGC members were encouraged to be involved in the institutions' internal continuous/routine quality assurance exercises as advisors and not to be externally driven in their quest for quality at the institutions.

d) Improvements for future TLQPR exercises.

Suggestions to improve the format of future exercises were discussed and the US model of a comprehensive institutional review encompassing research, teaching and learning and other overall aspects of a University was recommended.

Members opined that a credibility gap existed and recommended the UGC to state clearly its objectives and criteria. Its statement that "the exercise will inform funding" is still viewed as vague and obscure. Future review exercises should be institution specific, focusing on the institution's mission, history, culture and special characteristics of the institution. It should also include discipline and professional orientations at certain intervals so as to provide the depth necessary for the exercise.

For follow up activities, a small team of local and overseas members (other than those involved in the original exercise) was proposed so as to ensure a totally impartial view. The use of consultants was seen as unwise and viewed with suspicion given the experience elsewhere which has indicated the ineffectiveness of hiring consultancy firms like Coopers and Lybrand, and Peat Marwick to do the job.

Conclusion

The overall feedback seems to indicate that the level of consciousness about quality in teaching and learning has been raised to a limited extent. How effective it is will depend on how the UGC and the institutions continue to set about to sustain the impetus. It takes more than one round of the exercise to change the current thinking that research is still viewed as more important than teaching and learning because the rewards for research by the UGC are visible while those for teaching are still vague and non-committal. To sustain the change and move the institutions into a quality culture will require more commitment from everyone involved in tertiary education, along with the UGC. It is only when we see a proliferation of quality improvement research and discussions, publications, projects or seminars on the teaching and learning of subject disciplines by academics, and acceptance of teaching as a form of scholarship that we can say that the exercise has achieved its aims. The end result will depend on whether the institutions and UGC allow the process to lose its momentum for positive change.

The exercise unfortunately has overlooked the outcomes and value-added factors. Institutions with better reputations and resources would have an impact on the recruitment of more capable students than others. Institutions with greater intakes from the less-competitive sector of students and with fewer resources, but with equally good outcomes compared to the more privileged institutions should be gauged as having done a better job in teaching and learning quality. Without addressing the quality of outcomes and this value added factor, the exercise does not seem complete in itself. As Linke (1995) puts it, "the effectiveness of institutional quality assurance procedures is best indicated by the standard of performance outcomes rather than by the nature and perceived reliability of the procedures themselves." Perhaps the TLQPR exercise has over-emphasised procedures and mechanisms for quality assurance and paid little attention to quality itself in terms of both inputs and outputs.

Linke, R D (1995) Improving Quality Assurance in Australian Higher Education, Higher Education Management, Vol. 7 No.1 pp.49-62

8. Group Discussion Reports (cont'd)

Group 5 - Academic staff

Presented by Prof Dean Tjosvold, Chair Professor of Management, Lingnan College

This group of 20 academics directly involved in teaching discussed the four issues posed to it seriously and thoroughly. Groups of three and four debated the issues and then each group reported their findings to the larger group which in turn developed the groupís positions on each question. These positions were in turn the basis of the groupís oral report and this written one. Despite getting quickly down to the task, the group could have profitably used more time to consider the issues.

TLQPR accomplished its Aims?

The group reached a consensus that the TLQPR had accomplished its overall purpose. The tertiary institutions were now more focused on teaching and learning and this focus was made concrete by using teaching outcomes as part of the performance evaluation of academics. This effect of increased attention seems to have been particularly noticeable at the heavily research oriented institutions.

The institutions were also thought to be now more able to improve the quality review processes. They had put in places mechanisms for obtaining feedback and acting on it. However, the reviewís effects on actual improvement in quality teaching was less pronounced. Mention was made of the need to help instructors understand and use best teaching practices.

Group members generally were less confident that the institutions were able to hold themselves more accountable. How could accountability be defined and measured in ways that their institutions could use to document the effective use of resources? Group members also wanted their institutions to develop abilities so that they could carry out their own reviews and assessments.

Points

• more focus on teaching and learning, especially at ìresearch-orientedî institutions

• institutions more able to improve quality review processes

• doubts regarding increased ability to maintain accountability

• how can be review process be internalized within each institution?

TLQPR Impact on the Institutions?

The group concluded that the impact of the review was substantial. The review tended to reinforce the value of quality teaching at newer and smaller UGC institutions. The effects on HKU, CUHK and HKUST were more pronounced. The review at HKU was seen as quite instrumental for changing its cultures and developing several procedures to promote teaching.

However, the power of the review could also be channeled negatively. In one small group, two representatives thought the publicity surrounding the review had demoralized the staff by portraying them as ineffective teachers. The other person thought the publicity had encouraged and invigorated the teaching staff at his institution.

One pervasive influence of the review was that now student evaluations were institutionalized. The positive impact of the review needs to be evaluated in the context of increased committee work. Group members also wondered whether the pressure to keep up good teaching ratings may make instructors less willing to risk innovative methods that typically at first meet student resistance.

• at ìteachingî institutions, review reinforced and refined the teaching mission

• effects on value of quality teaching were more pronounced at HKU, CUHK, HKUST

• student feedback now institutionalized

• more time spent in committees

• publicity is powerful; it can invigorate but also demoralize.

Good Teaching Practices?

The academics in the group felt they and their colleagues were now more aware of good practices in teaching and especially in quality teaching processes. They also believed though that each institution must develop procedures that fit its culture and situation. One general condition is that students must be treated as partners as they give feedback, make suggestions, and help new teaching methods work.

• a more general awareness of good practices

• but each institution must develop its own quality processes

• students must be treated as partners.

Recommendations?

The group discussed as a whole recommendations for improvement. Members were willing to continue but time prevented us from a more thorough discussion. Concern was expressed that the ìwavesî of review left the institutions wondering and confused about their goals and primary objectives. Universities can be easily fragmented and its diverse members unclear of common goals and priorities. The various UGC reviews can augment these tendencies. The UGC should consider working with each institution to develop its mission and then relating to that institution based on this mission. This relationship would help institutions to be more focused and united.

More specific recommendations were to make sure to include postgraduates and to give enough time between reviews for the institutions to focus on ongoing activities. Some members also indicated that because teaching review processes were only a part of the mission of a university, outcomes of the review should not be tied to funding.

Recommendations are:

• reviews should be based on institutional ìmissionî

• UGC then could related to each institution based on this mission

• include postgraduates

• do not link funding with review outcomes.

Personal Reflection

The group discussion was quite useful for allowing academics an opportunity to voice their opinions and to listen and learn from others. They express their various views openly and constructively and I think learned from each other. I think others understood their own institution by comparing its experience with that of others.

I was impressed with the power of the review to impact a variety of feelings and attitudes. The review had gotten the attention of the member institutions and given its members in some cases a common ground for pride and in others a common feeling of failure.

8. Group Discussion Reports (cont'd)

Group 6 - Academic staff

Presented by Ms Maureen Tam, Educational Development Officer, Lingnan College

Members of Group 6 were mainly academics from higher education institutions and members of the UGC.

Members preferred to remain as a large group throughout the discussion because they thought that the four focus questions were related and could be looked at better in a holistic fashion.

However, the group agreed to organize the discussion in two major parts by grouping (a) and (b) under one strand, and (c) and (d) under another strand. Members felt that (a) and (b) are inseparable because one cannot say if the TLQPR has achieved its stated aims without making reference to its impact. Similarly for (d) and (e), the outcomes of the TLQPR were seen to be very much related to the subsequent suggestions for future improvements.

As a result of this, the report of Group 6 is therefore structured in two major parts:

1. Aims and impact of the TLQPR

Members of the group felt almost unanimously that an impact was made by the TLQPR. It has at least encouraged higher education institutions to take a good look within themselves to review the mechanisms and practices in place to support quality teaching and learning. Besides, the TLQPR has raised the importance of teaching and aroused the awareness of staff about the different dimensions of academic practice which should include teaching and research.

However, the group has reservations about how great the impact of the TLQPR has been to tip the balance between research and teaching. Members still felt that the incentive change caused by the TLQPR has not been great enough to counteract the preconception that research is a more important part of academic practice, especially when it comes to the appraisal of academic performance.

2. Outcomes and suggestions for improvement

Members felt that it was too early to give feedback on the TLQPR since this was the first time institutions had experienced a TLQPR visit. In connection to this, it was suggested that the TLQPR should not be linked with funding because of the need for institutions to learn from the experience. Perhaps it may be more appropriate if funding is related to the significant growth of institutions from their original base to a reasonable level of teaching and learning as a result of the TLQPR. In other words, members advocated a reward system rather than a penalty system.

With the assumption that people will learn more from positive feedback, it was suggested that feedback from the TLQPR panel should be positive and should commend the efforts already made.

For the feedback to be useful, it should be given at the unit level instead of being generalized across the whole institution. Members felt that the feedback was very much lost when it travelled from the top to bottom. Members were of the view that the TLQPR had used a two-prong approach which was found inadequate to address problems at both levels, i.e. at the unit and the institutional levels.

Last but not least, the group spent some time talking about the way the TLQPR reports were publicized. It was strongly felt by members that the UGC should make an effort to paint a more positive picture of the TLQPR so as to counteract the negative distortions reported by the Press.

A personal comment

If we are to evaluate the TLQPR, especially for (a) and (b), I think it may be useful to use a 'What-If' scenario.

'What if we didn't have the TLQPR? Would there be a difference?'

If the answer is 'Yes', it will then become easy for us to identify the impact, be it positive or negative.

From the discussions that Group Six had, I felt that members in general appreciated the TLQPR experience. They enjoyed the process of taking a closer look at how things are done in their institution, department and even within themselves as teachers. They enjoyed the interactions and sharing across the institutions, department and among individuals. However, some members expressed that they did not need the actual visit at the end. This is because the most valuable part of the TLQPR is not in the visit but in the process of getting the people and the institutions to engage in the review of their own teaching and learning practices.

I think this is very true to a large extent that the 'process' always outweighs the value of the 'outcomes' at the end. But how to make people go through the process without the visit remains a challenge to the TLQPR.

8. Group Discussion Reports (cont'd)

Group 7 - Academic staff

Presented by Dr Catherine Tang, Assoc Director, Educational Development Unit, The Hong Kong Polytechnic University

Members of Group 7 consisted of academic staff from eight tertiary institutions and members from the UGC. The focus of the discussion was on the following issues:

1. The extent to which the TLQPR exercise has achieved its stated aims

2. Impact of TLQPR on institutions (including unintended outcomes)

3. Outcomes in terms of identification of good practices for dissemination

4. Improvements to be made for future TLQPR exercises

In order to better involve group members in the discussion, they were asked to move into three small groups, discuss the issues and then report back for an open discussion and further comments.

It was observed that group members were very involved in the discussion, and most of them were very forthcoming in sharing their experience and in giving their opinions and comments on the TLQPR exercises. The following is a summary of the feedback and comments from group 7.

1. The extent to which the TLQPR exercise has achieved its stated aims

On the whole, the group thought that the TLQPR exercise had achieved its aims in assisting HKís tertiary institutions in focusing attention on teaching and learning as their primary mission. However, the group was of the opinion that it might be too early at this stage to ascertain whether or not the exercise has effectively assisted institutions in improving and maintaining accountability for the quality of teaching and learning. The extent to which this latter aim has been achieved might probably be better evaluated in the coming evaluation exercise and the next round of TLQPR.

2. Impact of TLQPR on institutions (including unintended outcomes)

The small group discussion focused more on the unintended outcomes which include:

• The exercise has increased the workload for colleagues such as the creation of various committees in relation to the exercise.

• The amount of time spent on the preparation for the visit was very considerable.

• Although the exercise has been perceived to have enabled institutions to focus on the quality of teaching and learning, some group members were not certain about the long term effects of the exercise in creating a quality culture within the institution. A lot would depend on what and how much follow-up action would be implemented.

3. Outcomes in terms of identification of good practices for dissemination

• The group commented that the list of good practices outlined in the report was useful and could assist institutions to focus on and learn from those practices which they do not currently have in place.

• The discussion on the good practices identified in the exercise has highlighted the importance of an appropriate and effective student feedback system, and the more effective and focused utilization of educational development type of units in promoting the quality of teaching and learning within the institutions.

4. Improvements to be made for future TLQPR exercises

The group members collectively would like to make the following recommendations for improvements for future TLQPR exercises:

• The exercise should be less formal. In view of the heavy involvement of staff time and effort, future exercises should not be conducted too frequently.

• Future TLQPR exercises should be more structured and systematic. More guidelines should be given especially for the type of paper/documentation required.

• Visits to individual units were perceived as the most important component of the exercise, and group members would like to suggest that more time should be given to unit visits. Some group members suggested a two-stage visit which would allow more time for interaction and discussion between panel members and institutions, and also for wash-up sessions.

• Although the TLQPR is an effective way of obtaining information on the quality processes in place in institutions, group members also suggested that alternative/additional channels through which institutions could provide feedback to UGC should be considered. Peer review was one such possible channel.

• Group members agreed that student involvement was an important component of the exercise. However, to better utilize this channel, the students should be better briefed about the aims and procedures of the exercise, and consideration should also be given to the language used during discussion with students. Better results might be achieved if discussion with students could be conducted in a language which the students are more comfortable with.

• Group members agreed that, in general, no names should be given in the TLQPR reports for very good reasons. However, as a form of recognition and encouragement, some group members suggested that units with good practices might be directly identified, while comments on ìpoorî practices should be kept general and anonymous.

• There was long discussion on the relationship between TLQPR results and funding. The group recognized that there is probably no simple solution, but the group would however like to suggest that funding should be linked with both process and outcome quality.

Personal Comment

The above is a summary of the discussion of group 7. I would like to add some comments on the seminar in general.

I found the seminar a very valuable exercise providing opportunity for soliciting feedback from various perspectives: the institution, individual academic staff, management and administrative staff, teaching support staff, and the UGC. Being able to give comments and suggestions about the exercise further enhances colleaguesí involvement, and makes them feel that they are not just the passive party in a top-down exercise. However, to reinforce this good feeling and ensure future cooperation, appropriate and substantial follow-up actions should be conducted rather than letting the exercise become just 'lip-service'.

The seminar was well organized and the organizing committee should be commended on the thought and effort they have put in. I personally enjoyed being the group leader, and I am grateful to the group members for their contributions. However, I do not think it was appropriate for group leaders to be invited to respond to questions in the open forum. To me, the role of a group leader is to facilitate small group discussion and report the groupís comments and suggestions, and we did just that happily. At least for myself, I do not think I am in the position to answer questions and challenges, and I did feel uncomfortable. However, notwithstanding this last comment, I would like to congratulate the UGC and the various committees involved on the success of the seminar.

8. Group Discussion Reports (cont'd)

Group 8 - Support Unit

Presented by Dr John Jones, Director, Education Development Unit, The Hong Kong Polytechnic University

The group worked on items (c) and (d), relating to "good practices" and "suggested improvements" respectively. This procedure was adopted, since it was clear that the TLQPR had had some impact (positive) on institutions, and had achieved at least some of its stated aims. It was felt that we could not usefully add to what had emerged in the morning session.

Four sub-groups worked separately on the questions, after which there was a report-back from each, and a brief discussion among the whole group. The major issues reported upon are outlined below.

(c) Comments in relation to "good practices" and their identification

• The whole exercise built up a culture in which self-evaluation was triggered, and there was less resistance (eventually) to the whole exercise. Furthermore, positive changes to practices were identifiable.

• It is easier to identify bad practices!

• "Good" practices can only be defined very broadly, and it is dangerous to lay down prescriptions for adoption by all institutions. Certainly, institutional differences should be allowed.

• Following from this last point was a warning that the lists contained in the Massy and French paper should be subjected to critique, as the paper was likely to be influential in setting directions for future exercises. Some specific comments in relation to this were the following.

Exemplary practices

- Too simplistic.

- Do not capture the significant elements of a quality culture.

- No reference to the 'scholarship' of teaching.

- Neglect the often problematic nature of e.g. student evaluations of teaching, Teaching Excellence Awards.

Effective Quality Processes

- Seem to be "text book" perspectives.

- No references to feedback from a broad range of stakeholders (e.g. staff, employers).

- No descriptions or discussions of how the move from "negative" to "positive" might occur.

- The meaning of "intellectual core" is unclear.

(d) Suggestions for ways in which future exercises might be improved

• Currently the objectives of the exercise are rather "soft"; they could usefully be clarified and more sharply focused.

• There is a need for early notification of the next exercise, and prior debate/discussion concerning the dimensions, and associated criteria, embodied in "good practice". (These would also stimulate/facilitate internal review).

• The next Review could usefully include assessment of "outcomes" which are selected by institutions (or units). More generally the next Review could encourage proactivity, and have institutions demonstrate what has been accomplished since the first exercise.

• General comments relating to preparation and documentation.

- Reduce the volume of documentation by e.g. providing clearer guidelines, using electronic formats.

- Explore "alternative formats" (including, e.g. surveying staff prior to visits).

- Provide clearer rationales for choice of units visited.

• General comments relating to Review visits

- Visit more units.

- Provide clearer objectives/formats for student input.

- Improve the arrangements for groups in such matters as guidelines, training, consistency.

- Include more practising teachers and local academics in Review teams.

• General comments relating to feedback Reports from the Review teams.

- Be clear about the conceptual distinction between

1. Unit feedback: for improvement.
2. Institutional feedback: for accountability and improvement.

- Avoid minority opinions in the Reports.

- Individual unit Reports could be more detailed, and forwarded promptly to the units concerned.

- The Reports should provide more adequate evidence and justification for the opinions and conclusions.

Personal Comment

It is very difficult to capture the complexity of the discussion, and the wide range of opinions and perspectives that emerged, in a bullet-point summary such as that above. My overall feeling is that the discussions were a useful beginning that should be continued over the whole period until the next TLQPR exercise, among as wide a range of stakeholders as possible. In that way a move toward a "culture" of ownership and self-regulation might occur. I sense that this is what most participants would favour i.e. a set of mechanisms for improvement and accountability which are developed in the context of each individual institution.

9. Seminar Summary

Speaker:    Professor Bill Massy

Prof Chan, Mr Imrie, colleagues

On behalf of the TLQPR Panel, I wish to thank all of you for all the very useful comments and feedback on the first round of TLQPRs.

I found the institutions' comments this morning both thoughtful and helpful. The subsequent group discussions were clearly lively and a number of useful points were raised. I am very pleased to note that colleagues generally felt that the first round of TLQPRs have achieved their stated aims; that the TLQPR exercise has re-affirmed the importance of quality in teaching and learning in the tertiary institutions; that the TLQPRs have helped identify and disseminate good practices; and that the TLQPRs have had a positive impact on the institutions although there are concerns on whether the impact would be sustained and whether the exercise would really effect changes to the quality of culture in the tertiary institutions.

Colleagues have debated whether teaching should be regarded the "primary mission" of higher education institutions, although I think this is more a matter of semantics than substance. There was considerable interest in how the results of TLQPRs would inform funding and some concern that the linkage would be so weak as to being unrecognisable. There was, however, general support for active follow-up such as dissemination of good practice and annual conferences. Some colleagues urged that the UGC should make an effort to draw a positive picture of the TLQPRs to balance the negative impression produced by selective reporting by the press.

There was also some very useful feedback on how future TLQPRs could be improved. Colleagues suggested that the criteria used in arriving at the conclusions and recommendations in the Review Reports should be made more transparent; future TLQPR exercises should be more structured and systematic; clear and consistent messages should be conveyed to the institutions in respect of the purpose and significance of the TLQPR exercise; lower ranking academic staff and students should be briefed more thoroughly about the aims and procedures of the TLQPR exercise; and the type of documentation to be submitted to the Review Panel could be simplified by providing more guidelines. As regards the unit level visits, more time should be spent on meetings with staff and students. Let me assure you that the UGC and the TLQPR Panel will take into account these helpful suggestions in future TLQPRs.

Thank you.

10. Closing Remarks

Speakers:    Prof Kenneth Young, The Chinese University of Hong Kong

              Prof Samuel Chan, The University of Hong Kong (Seminar Chairman)

Professor Kenneth Young

I would like to conclude by thanking all of you for your active participation in the Seminar which has resulted in a very useful exchange and fruitful discussion on a broad range of issues regarding the TLQPRs. The Seminar has provided constructive and invaluable comments on the TLQPR exercise, and the UGC will certainly use these comments to effect improvements for future TLQPR exercises. I believe that the UGC-funded institutions will join the UGC in promoting the need for continuous improvement of teaching and learning quality in the higher education sector in Hong Kong.

On behalf of the UGC I would also like to take this opportunity to express my warmest gratitude to Professor William Massy for his wisdom and most able leadership as the Chairman of the TLQPR Panel, and to all members of the TLQPR Panel for their contribution to the successful completion of the first round of the TLQPR exercise.

Professor Samuel Chan

It has been a long day but a very successful seminar. We have had stimulating discussions, good interactions, and constructive views which, I am sure and as Professor Kenneth Young has just mentioned, the UGC Quality Sub-Committee and TLQPR Panel will take note of in their planning for future TLQPR exercises. I also hope that such open forums can continue in the future, either in the form of a seminar or annual conference, as this clearly has served as an effective means of communication between the TLQPR Panel and the staff members of HEIs at all levels.

Before closing, it is now my duty and pleasure as Chairman to make some official acknowledgments to all those who have contributed to the success of the seminar. First, to the UGC for sponsoring the seminar and to Mr. French and his staff who have contributed behind the scene in organizing the seminar. To the Chairman and members of the TLQPR Panel, especially the external experts from overseas who have travelled a long way to Hong Kong for the TLQPR exercise. To Professor Kenneth Young for his stimulating Opening and Closing Remarks of the Seminar. To Professor Bill Massy for his presentations jointly with Mr. Nigel French on a review of the Hong Kong TLQPR Programme and his seminar summary. To all the speakers for their presentations. Finally, and most importantly, to the Organizing Committee, especially its Chairman Mr Brad Imrie, and also to the supporting staff from City University and camera team from Lingnan College, for their time and effort which has made today's seminar so successful.

I hope to see you all again next year!

Appendix 1 Participation According to Organisation

Participation According to Category

Appendix 2 Extracts from UGC Report

'HIGHER EDUCATION IN HONG KONG'

A Report by the University Grants Committee

October 1996

This Report is vitally important for the future of higher education in Hong Kong and represents the importance of partnership between the HEIs and the UGC. TLQPR is an example of this partnership and the following extracts are very relevant to the TLQPR Seminar.

Section D - The Quality of Teaching, Learning and Communication (p63)

Chapter 17 Quality Assurance (p66)

17.2 In 1993 ExCo confirmed that the responsibility for monitoring quality assurance at the UGC-funded institutions should rest with the UGC, a view which was supported by the institutions.

17.6 After much discussion between the UGC and its institutions, it has been agreed that, at least for the time being, quality assessment shall be eschewed in the consideration of teaching and learning (although it may be introduced for research (see paragraph 37.5)). Instead the Grants Committee will study process.

17.7 The use of Process Review as the main instrument of quality assurance is based on the assumption that the quality of graduate output depends principally on three factors: the quality of the student intake, the resources available, and the processes in place. The last is intentionally meant to be all-embracing; at one extreme it may include formal systems such as external examiners, while at the other it may try to take account of such elusive factors as the academic ambiance of an institution. The focus on process is appropriate because, of the three factors that determine the quality of the output, the available quality of the student intake is the responsibility of the school system and the overall level of resourcing is the responsibility of government, but the processes of teaching and learning are determined entirely by the higher education institutions themselves.

17.8 Process Reviews are naturally linked to an agenda for improvement, which must be the ultimate objective of any enquiry into quality. However, it is important to remember that the HEIs in Hong Kong have differing roles, missions and characteristics, and offer programmes in a wide variety of disciplines and in many styles; this diversity is a strength of the system that the UGC believes should be preserved and enhanced. It follows that the quality assurance processes that may be appropriate for one institution will almost certainly not be appropriate in their entirely for another. Any UGC quality assurance exercise must therefore avoid specifying a template or an ideal model against which to measure process, but must instead review the processes as they exist in each individual institution. These processes must, however, be comprehensive. They must control the quality of self-funding courses and those offered with overseas partners (see paragraph 2.10) as well as those fully funded by the UGC.

17.9 The UGC has always played a major role in the monitoring of quality assurance in its institutions. It has used a variety of mechanisms, including institutional and academic reviews, sectoral reviews, formal and informal visits and discussions at various levels. Additionally, the Committee has supported the institutions' own efforts in reviewing, maintaining, developing and enhancing the quality of their education provision. The Grants Committee has recently started a series of Teaching and Learning Quality Process Reviews. They are described in detail in Annex E. So far HKU, CUHK, HKUST, HKBU and LC have been reviewed. Review visits to CityU and PolyU are planned for January 1997. The reviews so far have gone well and the experience gained and examples of best practices are being shared among the institutions. The whole approach will be reconsidered by the Grants Committee after the first cycle of visits has been completed.

Annex E is attached for reference.

Annex E

Framework for the Teaching and Learning Quality Process Review

1. This Annex describes the framework used by the UGC in conducting and reporting on the first round of Teaching and Learning Quality Process Reviews. The framework, which emerged during preparation for the reviews and during the reviews themselves, provides a way of thinking about teaching and learning quality in tertiary education institutions. The UGC and its Review Panel will continue to develop the framework during subsequent TLQPR rounds.

TLQPR Goals

2. The goals of the TLQPRs are as follows:

1. to focus attention on teaching and learning as the primary mission of Hong Kongís tertiary institutions;

2. to assist institutions in their efforts to improve the quality of teaching and learning; and

3. to enable the UGC and the institutions to discharge their obligation to maintain accountability for the quality of teaching and learning.

The UGC, the TLQPR Panel, and the institutions share these goals and view the review as a collegial process.

TLQPR Methods

3. The TLQPR process begins with a preliminary visit by the Panel to each institution for the purposes of familiarizing staff with the purposes and methods of the Review, and the preparation by the institution of a twenty-page document describing its quality improvement and assurance processes. The Review visit lasts one and one-half days, which are utilized as follows:

• The first half day is devoted to three meetings: with the institutionís senior leadership, with the leadership plus academic staff associated with the quality improvement and assurance programme, and with students.

• The second half day involves meetings at the faculty level or with academic departments or quality programme support units. The Panel divides itself into six sub-groups for this purpose. Each subgroup meets with academic staff, students, and the leadership from two operating units, which allows visits to twelve units in all.

• The third half-day begins with a private session where the Panel formulates its preliminary impressions about the visit. The visit ends with a final meeting with the leadership and staff involved in quality assurance, where the preliminary impressions can be conveyed and discussed.

4. The TLQPR Panel consists of eighteen people: nine (including the Chair) members of the UGCís Quality Sub-Committee; two non-UGC overseas members with experience in higher education quality assurance; and one member from each of the seven Hong Kong institutions and starting from September 1996, one observer from the Hong Kong Institute of Education. The local members were designated by a larger TLQPR Consultative Committee of local-institution representatives, which has assisted the UGC in designing the TLQPR methodology. Because the TLQPR is collegial, the local Panel members participate fully in the reviews of their institutions.

Report Preparation

5. Report preparation proceeds in several stages. First, the institutionís self-analysis and discussion notes from the early plenary sessions are scrutinized for emergent themes and examples of exemplary and questionable practice. (The self-analysis summary uses the institutionís language to the extent possible.) The subgroup reports are similarly scrutinized, and a summary is prepared. The draft of this part of the report is reviewed by the Panel, and then by the institution for factual accuracy before submission to the UGCís Quality Sub-Committee. The ìAreas for Improvementî section is drafted concurrently and reviewed by the Panel, the Quality Sub-Committee, and the UGC before the final Report is transmitted to the institution. The institutions, in turn, have committed to make the reports public along with a statement describing the actions they plan to take by way of improvement.

TLQPR Dimensions

6. Teaching and learning quality can be viewed from two different perspectives. First come the teaching and learning processes themselves; in other words, the activities performed by academic and support-unit staff in performing their duties. Second come the methods by which institutions, faculties, departments, and similar units work to continuously improve teaching quality and assure themselves that the activities are appropriate and well executed.

7. The Review Panel recognizes that decisions with respect to both quality perspectives must be made by the institutions themselves, and that variety among and within institutions is necessary for an effective tertiary sector. The Panelís fundamental standard, therefore, lies not in specifying any particular approaches to teaching and learning quality, but rather in asking whether institutions and academic staff have given careful thought to both of the quality dimensions and whether they can articulate and defend the choices made.

Teaching and Learning Processes

8. Teaching and learning processes can be described in terms of the following five sub-processes, which form one dimension of the Panelís inquiry. Each sub-process is illustrated by questions which might be asked of an institution, a faculty, a department, or an individual staff member. However, the questions are presented by way of example only. The Panel does not presume that all the questions, or indeed any of them, are applicable in any particular situation. However, we do ask the institutions to organize their documentation in terms of the five sub-processes and we refer frequently to the five in our deliberations.

9. Curricular design: by what processes are programme curricula designed, reviewed, and improved? Some useful process elements follow:

1. Design inputs from the academic discipline, mainly staff-based

2. Design inputs from employers, feedback from current outcomes assessments, past students, professional bodies (where applicable), and other inputs dealing with ìfitness for useî

3. Integration mechanisms: how are these two kinds of inputs brought together? How are controversies resolved?

4. Faculty and institutional review mechanisms; what are they and how do they work?

5. External review mechanisms; e.g., visiting committees

10. Pedagogical design: by what process are the methods of teaching and learning decided and improved?

1. To what extent are pedagogical methods the subject of active consideration by staff, departments, faculties, etc.? Do staff spend sufficient time working together on these matters?

2. How broad is the definition of ìpedagogical methodî? For example, does it focus on learning as well as teaching? Does it integrate feedback about learning attainment with the delivery of academic content?

3. Degree of innovation in pedagogical method? Have the methods been changing over time? For example, have they been trending toward active as opposed to passive learning? Have they been taking sufficient advantage of information technology?

11. Implementation quality: processes related to how well the staff perform their teaching duties

1. How broad is the definition of ìteachingî? Does it include out-of-class student contact (including advising) and student assessment (including feedback about the assessments) as well as class contact?

2. What are the incentives for good teaching? What are the disincentives? (It is important to consider staff perceptions as well as the programmes themselves.)

3. How is teaching performance evaluated? (Possible mechanisms include self-evaluation, student evaluation, and peer evaluation.)

4. How are teaching evaluations utilized? For example, are they used in staff evaluation reviews? Are they shared among staff as part of a mutual-improvement process? Do they result in specific self-improvement efforts, such as utilization of teaching improvement centres?

12. Outcomes assessment: how do staff, departments, faculties, and the institution monitor student outcomes and link outcomes assessments to teaching and learning process improvement?î

1. Academic performance: for example, normed examinations and the use of external examiners

2. Other performance; for example, satisfaction as expressed in exit conferences, success in the job market

3. Feedback from past students, employers, etc.?

4. Are processes for working with students to help them achieve the desired teaching and learning outcomes in place and fully functioning?

13. Resource provision: are the human, technical, and financial resources needed for quality made available when and where needed?

1. Are the activities needed to achieve and assure teaching and learning quality given an appropriately high priority in the institutioní resource allocation process?

2. How do staff recruitment processes promote and safeguard the quality of teaching and learning?

3. How does the institutionís incentive and reward environment further the teaching and learning quality agenda?

4. To what extent does the institution offer technical assistance and training to staff who wish to improve their teaching and learning quality performance? To what extent are these resources utilized by staff?

Quality Improvement and Assurance

14. The Panel does not approach its task with any preconceived view of what an appropriate quality improvement and assurance programme should look like. On the contrary, we emphasize that the institutions should define their own processesóthat the Panelís job is to see whether such processes have in fact been defined and, once defined, whether they are being followed diligently. This view is consistent with the emergent international understanding of teaching quality in tertiary education, and with the fact that universities in Hong Kong are self-accrediting.

15. Certain broad areas of consideration for successful quality assurance have emerged from the Panelís queries and discussions, and these are presented below. We use them to convey examples of potentially useful quality assurance and improvement methods and to provide an organizing paradigm for our reports, but not as a template for judging an institutionís quality programme. However, the Panel does believe that to be fully effective, the assurance and continuous improvement of quality require a degree of self-consciousness and articulationówhich should be observable in the Review documents and site visit.

16. Quality programme framework: mission, vision, and policy statements pertaining to quality and quality assurance, expressed at the institutional level and at the level of faculties, departments, and other operating units. The framework provides a road map for individual and group action aimed at furthering and assuring teaching and learning quality.

17. Direct quality programme activities: undertaken by mainline teaching and administrative staff at the institutional level and at the level of faculties, departments, and other operating units. These activities are organized to assure quality levels and continuous quality improvement in the teaching and learning activities described in paragraph 6.

18. Quality programme support: funded special projects or activities undertaken by special teaching development or similar units organized to aid mainline teaching and administrative staff in performing their duties.

19. Values and incentives: the motivational environment for the improvement and assurance of teaching and learning qualityódriven by institutional, faculty or departmental values (intrinsic rewards) and formal or informal incentives (extrinsic rewards).

Sample Approaches

20. The following matrix presents some examples of how the four quality improvement and assurance methods can be applied to the five teaching and learning process dimensions. We observed each example in at least one of the institutions we visited, either centrally or at the level of a faculty, department, or other operating unit. The examples illuminate our framework for the TLQPR, but they are not intended to be definitive or prescriptive. Discussion of the various items and observations relevant to the individual institutions we visited is contained in the body of the report.

Sample Approaches