|
William F. Massy
Professor: Stanford University, U.S.A.
Member: Hong Kong University Grants Committee
International Conference on Quality Assurance and
Evaluation in Higher Education
Beijing, China
May 5-7, 1996
Contents
Abstract
To assure value for money in the higher education sector, the Hong Kong University Grants Committee (described in a companion paper) has embarked on a program of teaching and learning quality process reviews. This follows the implementation of a performance-based funding model and a research assessment exercise during 1994. This paper describes the next step in the UGC's quality assurance program, the introduction of "Teaching and Learning Quality Process Reviews" (TLQPRs) of the seven UGC-funded tertiary education institutions. After a brief introduction, the paper contrasts the three main types of quality assurance mechanisms used in higher education : accreditation, assessment, and audit. An accreditation system determines whether an institution or program meets minimum acceptable quality criteria : whether the institution's objectives are appropriate for the institutional or degree level, whether sufficient resources are available to meet the objectives, and whether the resources are used effectively to produce the desired outcomes. An assessment system evaluates the quality of specific activities : it makes graded judgements about academic quality levels rather than binary judgements relative to minimum acceptable standards. Assessment generally operates at the subject or program level because of the difficulty of making overall summative judgements. Assessments may be performed by parties external to the institutions themselves. In either case, but especially for external assessments, a choice must be made between accountability and improvement as goals because strong emphasis on accountability limits the exercise's effectiveness in stimulating improvement. An audit system (or a "review system," as it is described in Hong Kong) provides an external evaluation of internal quality assurance, assessment, and improvement systems. Audit does not evaluate quality itself : it focuses on the processes that are believed to produce quality and the methods by which academics assure themselves that quality has been attained. Audit takes place at the institutional level, though review teams will follow audit trails down to departments on a sampling basis. Audit methods currently are employed in the UK, the Netherlands and certain other European countries, and in Hong Kong. -
The paper will describe the TLQPRs currently being conducted in Hong Kong, and the actual experience at the first four institutions. The dimensions of teaching and learning quality to be reviewed include : To assure value for money in the higher education sector, the Hong Kong University Grants Committee (described in a companion paper) has embarked on a program of teaching and learning quality process reviews. This follows the implementation of a performance-based funding model and a research assessment exercise during 1994. This paper describes the next step in the UGC's quality assurance program, the introduction of "Teaching and Learning Quality Process Reviews" (TLQPRs) of the seven UGC-funded tertiary education institutions.
Curriculum design : by what processes are program curricula designed, reviewed, and improved? Padagogical design : by what process are the methods of teaching and learning decided and improved? Implementation quality : processes related to how well the staff perform their teaching duties. Outcome assessment : how do staff, departments, faculties, and the institution monitor student outcomes and link outcomes assessments to teaching and learning process improvement? Resource provision : are the human, technical, and financial resources needed for quality made available when and where needed?
Introduction
Long considered an intangible concept, quality is now being talked about as something that can be "measured", "monitored", and "managed." The concept of quality is not foreign to higher education. In fact, colleges and universities have long concerned themselves with quality (Dill 1992; 1982; Astin 1991; Ewell 1991; Dinham and Evans 1991; Kimball 1986; Webster 1986; 1983; Cameron 1985; Ben-David 1972; Crawford 1991; The Higher Education Funding Council of England 1994; Higher Education Quality Council 1994, 1995). Until recently, most attempts at quality assurance have focused on the reduction of variance both in inputs and outputs (Dill 1992). In the U.S., institutions have relied primarly upon market forces to minimize variance. However, it now is widely recognized that market forces alone cannot effectively police postsecondary educational quality.
Ideas about assuring educational quality are rooted in the business quality movement. Examples include: Deming methods, championed recently by David Dill (1995; 1992), who focuses specifically on "social capital," or social networks and horizontal mechanisms of communication; the (U. S.) Malcolm Baldrige National Quality Award, which has focused business on quality and quality and quality assurance methods and which now is being extended to non-profit enterprises including higher education (cf. Seymour 1994; Garvin 1991); process reengineering, which holds that the biggest obstacles to needed change are the reluctance of many in higher education to accept the notion of customers, the failure of most institutions to produce measurable outputs, and the lack of collegiality necessary to facilitate discussions about reengineering (cf. Massy and Wilger 1994, Kidwell and O'Brien 1995; Hammer 1990); and the "lean enterprise," as described by Womack and Jones (1994) as a group of individuals, functions, and separate organizations that come together to analyze and focus on a particular process to perfect it.
These approaches have in common certain fundamental characteristics including a focus on the customer, employee empowerment, a focus on process, the development of information systems, and an attempt to provide for continuous improvement. In the U.S., a recent Education of the States (ECS) report, Making Quality Count in Higher Education (1995) attempts to map quality assurance principles to academic processes in order to improve undergraduate education. The authors stress the importance of the organizational culture, a well-designed, coherent curriculum, and adequate feedback mechanisms. Recommended policy mechanisms include fiscal incentives such as performance and categorical funding and accountability mechanisms such as mandated assessment or quality process review.
This paper's main purpose is to describe quality process review as it has been developed and implemented in Hong Kong. But first let us consider the broader issues associated with higher education quality assurance and accountability.
Part I. Quality Assurance and Accountability
It is axiomatic that quality cannot be assured without accountability either from the marketplace or through some kind of external assessment or process review. It is clear also that accountability must reach all the way to the department or teaching group. Quality management principles imply that quality assurance has to be an integral part of the teaching and learning process, and that quality cannot be "inspected in" at the end. The implications are obvious: external and institution-level bodies cannot by themselves assure quality, though they can and should put in place the necessary conditions for quality assurance.
Quality must be the faculty's responsibility. However, contrary to the conventional culture in most institutions (Massy and Wilger 1995), quality must be a group responsibility rather than an individual one. Teams provide the most effective work setting for launching good quality assurance and improvement programs. Moreover, peer pressure provides the best way to police the performance of individual professors indeed, it is the only way when dealing with the fine structure of professional activities. The peer pressure must come from informed and involved colleagues who share a common stake in the outcome.
That educational quality is difficult for the market to judge is well known. While the possibility of effective market action cannot be ruled out, markets cannot provide the needed discipline without assistance. In other words, one needs some kind of organized assessment or process review process, not just attention to market signals. America's so-called "assessment movement" was initiated by a series of reports in the mid-1980s describing the shortcomings of the market as buttressed only by conventional accreditation: cf. Erwin and Knight (1995), Involvement in Learning (NIE 1984), Integrity in the College Curriculum (AAC 1985), and To Reclaim a Legacy (Bennett 1984); also Courts and McInerey (1993), Ewell (1991, 1988), and Heywood (1989). However, efforts by the states to mandate assessment have not proven successful, perhaps because the methods were viewed by institutions and faculty as too intrusive and lacking in validity.
Another approach focuses on the development of "performance indicators." These include measures of inputs (e.g., number of enrollments, number of faculty, revenues, expenditures), outputs (credits and degrees granted, research publications), and overt measures of quality (admissions selectivity, fellowships and prizes won, peer or press evaluations). But while helpful, such measures are too crude to serve as the primary vehicle for achieving accountability. In fact, an over-reliance on input measures like expenditures per student can drive up costs and distort one's view of quality (Zemsky and Massy 1995).
More fine-grained techniques are needed. These would appear to fall into one of three categories: accreditation, assessment (European style), and process review (also called "quality process audit"). European-style assessment, described below, generally takes a broader approach than is typically described in the U.S. literature. Accreditation tends to be similar across national boundaries, and the quality process review described in this paper is a relatively recent phenomenon.
Accreditation and Assessment
Accreditation determines whether an institution or a program meets threshold quality criteria. The methodology generally utilizes a combination of performance indicators, self-study, and peer review. As described above, performance indicators provide quantitative data on resources and performance. Self-studies represent an institution's evaluation of its own performance in relation to standards and its own particular aspirations based on both PIs and subjective factors. Peer review relies on the experience of outside experts who visit the campus and form their own opinions about performance in relation to standards.
The general characteristics of accreditation are as follows.
Accreditation deals exclusively with education, either at the institutional or program level. Program-level accreditation is most common in professional fields like accounting, business, law, and engineering and for institutions offering degrees below the bachelors' level. Institutional accreditation is most common for general undergraduate programs. Accreditation is criterion-referenced, That is, it compares observed performance against pre-set standards usually determined by the accrediting agency. Accreditation evaluates whether an institution's objectives are appropriate for the degree level in question as well as its implementation of the objectives. Typical implementation questions include whether sufficient resources are available to meet the objectives and whether the resources are used effectively to produce the desired outcomes. Because accreditation performs a certification function, it must be performed by an agency external to the institution itself: e.g., an arm of government or a professional association, a consortium of higher education institutions, or a free standing entity.
Accreditation assures stakeholders that minimum standards are being met and allows parties not otherwise familiar with the institution to evaluate the efficacy of credits and degrees against a known baseline. However, most systems have tried to graft an improvement agenda on to the traditional certification function. But whether the two can coexist effectively is open to debate. Cycle times represent one potential problem. (Another problem, discussed below, pertains to the publication of results.) Accreditation cycles are typically long (ten years for U.S. institutional accreditation) unless serious problems are uncovered. But while long cycle times are consistent with quality certification, they may be longer than optimal for stimulating improvement. Indeed, they may be dysfunctional for improvement in that, once accredited, an institution may sigh with relief and be content to continue the status quo until the next accreditation looms.
The final outcome of accreditation, whether the institution meets threshold standards, is always published. Such publication is necessary for accreditation to perform its certification function. However, the report itself may be withheld to avoid adversarial relationships and, thus, to protect data acquisition and enhance accreditation's improvement agenda. This view is strongly held by the U. S. institutional accreditation agencies, but increasing numbers of commentators argue that confidentiality undermines an institution's ability to confront the need for improvement.
Assessment evaluates the quality of specific educational activities. In other words, assessment goes beyond accreditation's binary rulings about certification to make graded judgments about academic quality levels that fall above threshold standards. Assessment generally follows accreditation in using a combination of performance indicators, self study, and peer review. Some assessment systems have been criticized for relying too heavily on performance indicators, and now there appears to be a consensus that assessment should include site visits by peer reviewers.
The general characteristics of assessment are as follows.
Assessment generally operates at the subject or program level. The UK makes separate subject-level assessments for teaching and research (cf. Davies 1995; Massy and El-Khawas 1996; HEFCE 1992a, 1992b). It is hard to see how one can make effective institution-level quality assessments without first assessing the individual programs. Assessment can be organized by an external agency (such as government), an institutional consortium, or by the institutions themselves. Assessment tends not to be as strongly linked to external criteria as is accreditation. For example, teaching quality is generally defined relative to an institution's mission, not according to some "gold standard" of academic excellence. However, the UK's research assessment methodology has been criticized for overemphasizing peer-reviewed publication rather than applied research that is more consistent with some institutions' missions.
Accountability generally provides the proximate motivation for externally-organized assessment, while improvement provides the motivation for institutionally-initiated assessment. However, accountability goals generally include improvement as an indirect benefit and internal assessment processes can be used an element of accountability (see below).
Accountability generally requires that the results of assessment be published, and that the assessment results be expressed in a way that permits comparisons among institutions. Media interest leads to the production and publication of league tables, whether or not these were originally contemplated by the assessment agency. These considerations tend to work against externally-driven assessment's improvement goals. For similar reasons, many commentators believe that teaching assessment scores should not be used as direct inputs to the funding process. It is argued, for example, that direct utilization motivates institutions to view the process adversarially, perhaps to the point of withholding data. (This problem is less severe for research assessments, since most research output already is in the public domain.) However, most people believe that the assessment results should inform funding judgments indirectly if for no other reason that funding agencies have an obligation to take all available information into account when making decisions.
Externally-organized assessment cycle times tend to be in the range of five to ten years. They tend to be somewhat shorter than accreditation cycle times, but arguably too long to instill a culture of continuous improvement. Assessment's long cycle times are dictated by the cost of site visits and the large number of assessment units (which can approach the number of subjects times the number of institutions). Assessments organized by the institutions themselves are less affected by such limitations, so their cycle times can be shorter.
Quality Process Review
As implemented in the UK, Hong Kong, The Netherlands, and in a few other countries, the quality process review is an externally-driven meta-analysis of internal quality assurance, assessment, and improvement systems. Unlike assessment, these reviews do not evaluate quality itself. Instead, they focus on the processes that are believed to produce quality and the methods by which institutions, faculties, and departments assure themselves that quality has been attained. (See Higher Education Quality Council, 1994, for a comprehensive description of how the process is being implemented in the UK.)
Quality process reviews are founded on the principle that good people working with sufficient resources and according to good processes will produce good results, but that faulty processes will prevent even good people and plentiful resources from producing optimal outcomes. Quality process reviews generally take place at the institutional level, though there is nothing to prevent subject-level audits. In fact, The Netherlands Higher Education Inspectorate follows up on the implementation of institutionally-organized subject-level assessment results with what could be argued are quality process reviews.
Quality process reviews are inherently less demanding than assessment, for the following reasons:
One can determine what professors do and how they do it more easily than one can determine the actual quality of teaching and learning outcomes. Most assessments include the evaluation of both process outcome, which multiplies the dimensions of complexity. Adopting the quality process review sidesteps the need to predetermine output measures and standards; instead, ones asks whether the institution has done so and whether the institution's choices appear reasonable in light of its mission.
Whereas output standards may well differ across institutions, the standards for judging processes appear to be much less variable. Indeed, core process attributes like customer focus, effective performance feedback, and collegial consideration of improvement possibilities would seem to be universally desirable. Furthermore, process review cycle times can be significantly less than those for assessment because the reviews are inherently simpler. Process review reports are always made public. However, details based on self-studies, internally-initiated assessments, and plans for improvements need not be disclosed.
Some institution-level quality process reviews focus mainly on the formalities of quality assurance: for example, on policy statements, rules and procedures, guidance notes, and meeting minutes. This is consistent with the "audit" terminology which is still in use in the UK. It also provokes the main criticism of the audit approach: that it is bureaucratic rather than substantive. But while an emphasis on formalities may seem natural if an institution has not internalized the quality culture, it is by no means necessary. In fact, a process review can and should concentrate upon the vitally important questions of what professors do, how they do it, and how they acquire the performance feedback needed for continuous improvement. By following audit trails to look at faculty and departmental records and interview staff and students at the subject level, quality process reviewers can uncover the important informal dimensions of quality assurance, and whether the staff have internalized the institution's quality improvement and assurance processes.
Tradeoffs Between Accountability and Improvement
Considerable tension exists between the accountability and improvement goals of external quality assurance programs. The problem arises mainly in assessment, although it can arise with process reviews as well. The nature of the conflict is illuminated by the following polar scenarios. (The scenarios are somewhat overdrawn in order to highlight the nature of the conflict; however, readers may recognize elements of reality.)
Accountability is dominant; improvement is ineffectual
Assessment is performed directly by a government unit: peers are not involved or are not sufficiently involved to provide credibility; points of view from outside the academy dominate.
The system relies heavily on performance indicators, which may raise but do not answer quality questions. Institutions are concerned that some performance indicators and assessors' judgments do not reflect academic values and the realities of academic performance or, worse, that they reflect political and ideological positions. Everything is published; "sunshine rules" prohibit private communication between assessors and institutions. There is a direct linkage to funding, with little opportunity for institutions to discuss and interpret outcomes.
The resulting "compliance culture" will reflect a "we-they" attitude, rather than a joint ownership of quality assurance and improvement. Information will be withheld so far as possible, and efforts will be made to "maximize the indicators" while defending the status quo rather than seeking improvement.
Ironically, the attempt to maximize accountability actually reduces the amount of real accountability in the system. The effects on improvement are disastrous.
Improvement is dominant; accountability is surpressed
Assessment has been captured by institutions; peers are dominant; few points of view from outside the academy are considered. Performance indicators are eschewed on the ground that nothing of importance can be measured quantitatively. Judgments reflect traditional academic values as interpreted by a small group of peers all of whom have "made it" according to the traditional criteria. Nothing is published, ostensibly to avoid the problem of data withholding; all communications between assessors and the institution, other than (perhaps) whether the assessment has produced a passing grade, are confidential. There is no linkage between the assessment results and funding, or with anything else of value to the institution. The resulting highly collegial process may well be praised for its sensitivity to academic values. Participants will own the process, and they may try to affect change incrementally without challenging the conventional wisdom. However, highly collegial processes fail to produce a sense of urgency, and they often inhibit efforts to identify problems and search for solutions that may challenge established norms. Indeed, such processes may actually reduce accountability by periodically certifying the effectiveness of the status quo. Ironically, the attempt to maximize improvement fails to elicit change outside the narrow band of conventional wisdom. It can actually inhibit improvement and undermine accountability.
Anchoring the two ends of the accountability-improvement spectrum makes clear that a tradeoff between the two is required. The idea of a tradeoff suggests that one does not have to make an unequivocal choice between the two goals: one can select a portfolio of QA process elements which, when combined into an overall system, balances the advantages and risks associated with each goal. That is what the Hong Kong UGC is working to accomplish.
Part II. The Hong Kong Quality Process Reviews
To assure value for money in the higher education sector, the Hong Kong University Grants Committee (described in a companion paper) has embarked on a program of quality process reviews. This section describes the UGC's "Teaching and Learning Quality Process Reviews" (TLQPRs), as conducted at The Hong Kong University and the Chinese University of Hong Kong during January of 1996. Similar reviews will be conducted at the other five UGC-funded institutions during the next eight months.
The goals of the TLQPRs are to focus attention on teaching and learning, to assist institutions in their efforts to improve teaching and learning quality, and to enable the UGC and the institutions to discharge their obligation to maintain accountability for quality. These goals are shared by the UGC and the institutions. The TLQPR program follows the implementation of a performance-based funding model and a research assessment exercise during 1994. The reviews are conducted by a TLQPR Panel consisting of eight UGC members, fourteen members from the seven UGC institutions, and two overseas experts on higher education quality assurance. All members participate in all the Panel's activities, including visits to their own institutions.
TLQPR Methods
The TLQPR begins with a preliminary visit by the Panel to each institution for the purposes of familiarizing staff with the purposes and methods of the review, and the preparation by the institution of a twenty-page document describing its quality improvement and assurance processes. The review visit lasts one and one-half days, which are utilized as follows:
The first half day is devoted to three meetings: with the institution's senior leadership, with the leadership plus academic staff associated with the quality improvement and assurance program, and with students. The second half day involves meetings at the faculty level or with academic departments or quality program support units. The Panel divides itself into six sub-groups for this purpose. Each subgroup meets with academic staff, students, and the leadership from two operating units, which allows visits to twelve units in all. The third half-day begins with a private session where the Panel formulated its preliminary impressions about the visit. The visit ends with a final meeting with the leadership and staff involved in quality assurance, where the preliminary impressions can be conveyed and discussed.
Report preparation proceeds in several stages. First, the institution's self-analysis and discussion notes from the early plenary sessions are scrutinized for emergent themes and examples of exemplary and questionable practice. (The self-analysis summary uses the institution's language to the extent possible.) The subgroup reports are similarly scrutinized, and a summary is prepared. The draft of this part of the report is reviewed by the Panel, and then by the institution for factual accuracy before submission to the UGC's Quality Subcommittee. The "Areas for Improvement" section is drafted concurrently and reviewed by the Panel, the Quality Subcommittee, and the UGC before the final Report is transmitted to the institution. The institutions, in turn, have committed to make the reports public along with a statement describing th actions they plan to take by way of improvement.
TLQPR Dimensions
The Panel has decided to view teaching and learning quality from two different perspectives. First come the teaching and learning processes themselves: that is, the activities performed by academic and support-unit staff in performing their mainline duties. Second come the methods by which institutions, faculties, departments, and similar units work to continuously improve teaching quality and assure themselves that the mainline activities are maximally appropriate and well executed.
The Review Panel recognizes that decisions with respect to both quality dimensions must be made by the institutions themselves, and that variety among and within institutions is necessary for an effective tertiary sector. The Panel's fundamental standard, therefore, lies not in specifying any particular approaches to teaching and learning quality, but rather in asking whether institutions and academic staff have given careful thought to both of the quality dimensions and whether they can articulate and defend the choices made.
Teaching and Learning Processes
Teaching and learning processes can be described in terms of the following five sub-processes, which form one dimension of the Panel's inquiry. Each sub-process is illustrated by questions which might be asked of an institution, a faculty, a department, or an individual staff member. However, the questions are presented by way of example only. The Panel does not presume that all the questions, or indeed any of them, are applicable in any particular situation. However, we do ask the institutions to organize their documentation in terms of the five sub-processes and we refer frequently to the five in our deliberations.
-
Curriculum design: by what processes are program curricula designed, reviewed, and improved? Some useful process elements follow:
Design inputs from the academic discipline, mainly staff-based Design inputs from employers, feedback from current outcomes assessments, past students, professional bodies (where applicable), and other inputs dealing with "fitness for use" Integration mechanisms: how are these two kinds of inputs brought together? How are controversies resolved? Faculty and institutional review mechanisms; what are they and how do they work? External review mechanisms; e.g., visiting committees
-
Pedagogical design: by what process are the methods of teaching and learning decided and improved?
To what extent are pedagogical methods the subject of active consideration by staff, departments, faculties, etc.? Do staff spend quality time working together on these matters? How broad is the definition of "pedagogical method"? For example, does it focus on learning as well as teaching? Does it integrate feedback about learning attainment with the delivery of academic content? Degree of innovation in pedagogical method? Have the methods been changing over time? For example, have they been trending toward active as opposed to passive learning? Have they been taking sufficient advantage of information technology?
-
Implementation quality: processes related to how well the staff perform their teaching duties
How broad is the definition of "teaching"? Does it include out-of-class student contact (including advising) and student assessment (including feedback about the assessments) as well as class contact? What are the incentives for good teaching? What are the disincentives? (It is important to consider staff perceptions as well as the programs themselves.) How is teaching performance evaluated? (Possible mechanisms include self-evaluation, student evaluation, and peer evaluation.) How are teaching evaluations utilized? For example, are they used in staff evaluation reviews? Are they shared among staff as part of a mutual-improvement process? Do they result in specific self-improvement efforts, such as utilization of teaching improvement centers?
-
Outcomes assessment: how do staff, departments, faculties, and the institution monitor student outcomes and link outcomes assessments to teaching and learning process improvement?"
Academic performance: for example, normed examinations and the use of external examiners Feedback from past students, employers, etc.? Are processes for working with students to help them achieve the desired teaching and learning outcomes in place and fully functioning?
-
Resource provision: are the human, technical, and financial resources needed for quality made available when and where needed?
Are the activities needed to achieve and assure teaching and learning quality given an appropriately high priority in the institution's resource allocation process? How do staff recruitment processes promote and safeguard the quality of teaching and learning? How does the institution's incentive and reward environment further the teaching and learning quality agenda? To what extent does the institution offer technical assistance and training to staff who wish to improve their teaching and learning quality performance? To what extent are these resources utilized by staff?
Quality Improvement and Assurance
The Panel does not approach its task with any preconceived view of what an appropriate quality improvement and assurance program should look like. On the contrary, we emphasize that the institutions should define their own processes that the Panel's job is to see whether such processes have in fact been defined and, once defined, whether they are being followed diligently. This view is consistent with the emergent international understanding of teaching quality in tertiary education, described in Part I, and with the fact that universities in Hong Kong are self-accrediting.
Certain broad areas of consideration for successful quality assurance have emerged from the Panel's queries and discussions. These are presented below. We use them to convey examples of potentially useful quality assurance and improvement methods and to provide an organizing paradigm for our reports, but not as a template for judging an institution's quality program. However, the Panel does believe that to be fully effective, the assurance and continuous improvement of quality require a degree of self-consciousness and articulation which should be observable in the Review documents and site visit..
Quality program framework: mission, vision, and policy statements pertaining to quality and quality assurance, expressed at the institutional level and at the level of faculties, departments, and other operating units. The framework provides a roadmap for individual and group action aimed at furthering and assuring teaching and learning quality. Direct quality program activities: undertaken by mainline teaching and administrative staff at the institutional level and at the level of faculties, departments, and other operating units. These activities are organized to assure quality levels and continuous quality improvement in the teaching and learning activities described earlier. Quality program support: funded special projects or activities undertaken by special teaching development or similar units organized to aid mainline teaching and administrative staff in performing their duties. Values and incentives: the motivational environment for the improvement and assurance of teaching and learning quality driven by institutional, faculty or departmental values (intrinsic rewards) and formal or informal incentives (extrinsic rewards).
Sample Approaches
Figure 1 presents some examples of how the four quality improvement and assurance methods can be applied to the five teaching and learning process dimensions. The Panel observed each example in at least one of its first-round visits, either centrally or at the level of a faculty, department, or other operating unit. The examples illuminate the TLQPR framework, but they are not intended to be definitive or prescriptive. However, they do provide a good representation of the kinds of practices observed.
Conclusions
While the Hong Kong Teaching and Learning Quality Process Reviews still represent work in process, we have gained sufficient experience to be confident that the methodology is achieving the desired objectives. As hoped, progress on the institutions' self-improvement agenda provides an important initial benefit. The TLQPR also will provide the UGC with a set of priorities to guide its discussions with institutions about teaching and learning quality. We expect the institutions' leadership and staff to vigorously pursue the same quality goals as the UGC. However, should problems arise, the UGC will be in a better position to promulgate its accountability agenda than it would have been without the TLQPR.
In general, the Panel was satisfied that sufficient improvement and assurance processes are in place and in prospect in the first two institutions visited to warrant a satisfactory degree of confidence about the delivered quality of teaching and learning in Hong Kong. Particularly satisfying were the initiatives put in place by the institutions given the prospect of the TLQPR. The event of an impending review provided an impetus for increased or in some situations de nova attention to elements of the teaching and learning quality paradigm described above. Given proper followup, such initiatives are expected to produce significant behavior changes. And while such behavior shifts do not guarantee desired changes in the academic culture, they represent a good starting point.
The Panel also identified a significant number of areas where improvement is needed. The lacuna were particularly apparent when we looked at the degree of understanding, buyin, and followthrough on institutional initiatives at the faculty and departmental level. (The Panel found a great deal to commend at the faculty and departmental levels, however.) Bringing these slippages in accountability to the attention of the institutions' leadership and staff represents the necessary first step toward improvement.
In summary, it can safely be said that Hong Kong's first teaching and learning quality process review is off to a successful start. By focusing on substance instead of formalities, the TLQPR Panel has mitigated the criticisms leveled against the UK's quality process audits. The initial response to the visits has been good, and the reviews appear to be meeting the UGC's short- and long-term goals. The Panel will update and adapt the review program as experience is gained and conditions change. Only time will tell how much the TLQPR will spur improvement and enable accountability, but the Panel and the UGC are optimistic about the prospects.
|
