Annex E - Framework for Teaching and Learning Quality Process Reviews
This Annex describes the framework used by the UGC in conducting and reporting on the first round of Teaching and Learning Quality Process Reviews. The framework, which emerged during preparation for the reviews and during the reviews themselves, provides a way of thinking about teaching and learning quality in tertiary education institutions. The UGC and its Review Panel will continue to develop the framework during subsequent TLQPR rounds.
TLQPR Goals
The goals of the TLQPRs are as follows:
to focus attention on teaching and learning as the primary mission of Hong Kong's tertiary institutions;
to assist institutions in their efforts to improve the quality of teaching and learning; and
to enable the UGC and the institutions to discharge their obligation to maintain accountability for the quality of teaching and learning.
The UGC, the TLQPR Panel, and the institutions share these goals and view the review as a collegial process.
TLQPR Methods
The TLQPR process begins with a preliminary visit by the Panel to each institution for the purposes of familiarizing staff with the purposes and methods of the Review, and the preparation by the institution of a twenty-page document describing its quality improvement and assurance processes. The Review visit lasts one and one-half days, which are utilized as follows:
The first half day is devoted to three meetings: with the institutionÕs senior leadership, with the leadership plus academic staff associated with the quality improvement and assurance programme, and with students.
The second half day involves meetings at the faculty level or with academic departments or quality programme support units. The Panel divides itself into six sub-groups for this purpose. Each subgroup meets with academic staff, students, and the leadership from two operating units, which allows visits to twelve units in all.
The third half-day begins with a private session where the Panel formulates its preliminary impressions about the visit. The visit ends with a final meeting with the leadership and staff involved in quality assurance, where the preliminary impressions can be conveyed and discussed.
The TLQPR Panel consists of eighteen people: nine, (including the Chair) members of the UGC's Quality Subcommittee; two non-UGC overseas members with experience in higher education quality assurance; and one member from each of the seven Hong Kong institutions. The local members were designated by a larger TLQPR Consultative Committee of local-institution representatives, which has assisted the UGC in designing the TLQPR methodology. Because the TLQPR is collegial, the local Panel members participate fully in the reviews of their institutions.
Report Preparation
Report preparation proceeds in several stages. First, the institution's self-analysis and discussion notes from the early plenary sessions are scrutinized for emergent themes and examples of exemplary and questionable practice. (The self-analysis summary uses the institution's language to the extent possible.) The subgroup reports are similarly scrutinized, and a summary is prepared. The draft of this part of the report is reviewed by the Panel, and then by the institution for factual accuracy before submission to the UGC's Quality Sub-Committee. The "Areas for Improvement" section is drafted concurrently and reviewed by the Panel, the Quality Sub-Committee, and the UGC before the final Report is transmitted to the institution. The institutions, in turn, have committed to make the reports public along with a statement describing the actions they plan to take by way of improvement.
TLQPR Dimensions
Teaching and learning quality can be viewed from two different perspectives. First come the teaching and learning processes themselves; in other words, the activities performed by academic and support-unit staff in performing their duties. Second come the methods by which institutions, faculties, departments, and similar units work to continuously improve teaching quality and assure themselves that the activities are appropriate and well executed.
The Review Panel recognizes that decisions with respect to both quality perspectives must be made by the institutions themselves, and that variety among and within institutions is necessary for an effective tertiary sector. The Panel's fundamental standard, therefore, lies not in specifying any particular approaches to teaching and learning quality, but rather in asking whether institutions and academic staff have given careful thought to both of the quality dimensions and whether they can articulate and defend the choices made.
Teaching and Learning Processes
Teaching and learning processes can be described in terms of the following five sub-processes, which form one dimension of the Panel's inquiry. Each sub-process is illustrated by questions which might be asked of an institution, a faculty, a department, or an individual staff member. However, the questions are presented by way of example only. The Panel does not presume that all the questions, or indeed any of them, are applicable in any particular situation. However, we do ask the institutions to organize their documentation in terms of the five sub-processes and we refer frequently to the five in our deliberations.
Curriculum design: by what processes are programme curricula designed, reviewed, and improved? Some useful process elements follow:
Design inputs from the academic discipline, mainly staff-based
Design inputs from employers, feedback from current outcomes assessments, past students, professional bodies (where applicable), and other inputs dealing with "fitness for use"
Integration mechanisms: how are these two kinds of inputs brought together? How are controversies resolved?
Faculty and institutional review mechanisms; what are they and how do they work?
External review mechanisms; e.g., visiting committees
Pedagogical design: by what process are the methods of teaching and learning decided and improved?
To what extent are pedagogical methods the subject of active consideration by staff, departments, faculties, etc.? Do staff spend sufficient time working together on these matters?
How broad is the definition of "pedagogical method"? For example, does it focus on learning as well as teaching? Does it integrate feedback about learning attainment with the delivery of academic content?
Degree of innovation in pedagogical method? Have the methods been changing over time? For example, have they been trending toward active as opposed to passive learning? Have they been taking sufficient advantage of information technology?
Implementation quality: processes related to how well the staff perform their teaching duties
How broad is the definition of "teaching"? Does it include out-of-class student contact (including advising) and student assessment (including feedback about the assessments) as well as class contact?
What are the incentives for good teaching? What are the disincentives? (It is important to consider staff perceptions as well as the programmes themselves.)
How is teaching performance evaluated? (Possible mechanisms include self-evaluation, student evaluation, and peer evaluation.)
How are teaching evaluations utilized? For example, are they used in staff evaluation reviews? Are they shared among staff as part of a mutual-improvement process? Do they result in specific self-improvement efforts, such as utilization of teaching improvement centers?
Outcomes assessment: how do staff, departments, faculties, and the institution monitor student outcomes and link outcomes assessments to teaching and learning process improvement?"
Academic performance: for example, normed examinations and the use of external examiners
Other performance; for example, satisfaction as expressed in exit conferences, success in the job market
Feedback from past students, employers, etc.?
Are processes for working with students to help them achieve the desired teaching and learning outcomes in place and fully functioning?
Resource provision: are the human, technical, and financial resources needed for quality made available when and where needed?
Are the activities needed to achieve and assure teaching and learning quality given an appropriately high priority in the institutionÕs resource allocation process?
How do staff recruitment processes promote and safeguard the quality of teaching and learning?
How does the institution's incentive and reward environment further the teaching and learning quality agenda?
To what extent does the institution offer technical assistance and training to staff who wish to improve their teaching and learning quality performance? To what extent are these resources utilized by staff?
Quality Improvement and Assurance
The Panel does not approach its task with any preconceived view of what an appropriate quality improvement and assurance programme should look like. On the contrary, we emphasize that the institutions should define their own processes;that the Panel's job is to see whether such processes have in fact been defined and, once defined, whether they are being followed diligently. This view is consistent with the emergent international understanding of teaching quality in tertiary education, and with the fact that universities in Hong Kong are self-accrediting.
Certain broad areas of consideration for successful quality assurance have emerged from the Panel's queries and discussions, and these are presented below. We use them to convey examples of potentially useful quality assurance and improvement methods and to provide an organizing paradigm for our reports, but not as a template for judging an institution's quality programme. However, the Panel does believe that to be fully effective, the assurance and continuous improvement of quality require a degree of self-consciousness and articulation; which should be observable in the Review documents and site visit.
Quality programme framework: mission, vision, and policy statements pertaining to quality and quality assurance, expressed at the institutional level and at the level of faculties, departments, and other operating units. The framework provides a road map for individual and group action aimed at furthering and assuring teaching and learning quality.
Direct quality programme activities: undertaken by mainline teaching and administrative staff at the institutional level and at the level of faculties, departments, and other operating units. These activities are organized to assure quality levels and continuous quality improvement in the teaching and learning activities described in paragraph 6.
Quality programme support: funded special projects or activities undertaken by special teaching development or similar units organized to aid mainline teaching and administrative staff in performing their duties.
Values and incentives: the motivational environment for the improvement and assurance of teaching and learning quality‹driven by institutional, faculty or departmental values (intrinsic rewards) and formal or informal incentives (extrinsic rewards).
Sample Approaches
The following matrix presents some examples of how the four quality improvement and assurance methods can be applied to the five teaching and learning process dimensions. We observed each example in at least one of the institutions we visited, either centrally or at the level of a faculty, department, or other operating unit. The examples illuminate our framework for the TLQPR, but they are not intended to be definitive or prescriptive. Discussion of the various items and observations relevant to the individual institutions we visited is contained in the body of the report.
 |