On the Future of the TLQPR

William F. Massy

Workshop on Preparing for the Next Round of TLQPRs

University Grants Committee: Hong Kong, April 8, 2000

Thank you for joining us today. I know that assuring and improving educational quality is a high priority for each of you, as it is for the UGC and your institutions.

I'm pleased that the CHEPS team concluded that "TLQPR was the right instrument at the right time" for Hong Kong and recommended that the UGC should "continue the campaign for quality". Our next speaker will surely elaborate on that. I would like to briefly sketch some of the things I've learned, since the first TLQPR round, from my research at the National Center for Postsecondary Improvement (NCPI) at Stanford University in the U.S.A. You may be interested to know that NCM and Professor David Dill (from the University of North Carolina and a member of the CHEPS team) have introduced the TLQPR principles in the United States, and that at least two of our seven regional accreditation agencies are considering their adoption as part of institutional accreditation.

What we did here in Hong Kong can now be described as stimulating "Educational Quality Work" (EQW), the system of activities within universities that improves and assures educational quality. The term was invented by the Swedish National Agency for Higher Education as they performed their first round of reviews---which, by the way, used many of the same principles as our TLQPR. EQW focuses on performance feedback and the organizational processes needed to act on the feedback. EQW should not be confused with teaching and learning itself. It is the "feedback and control system" that guides teaching and learning.

Quality work takes place mainly at the department and program level, by professors committed to improving the curriculum, finding better ways to teach and learn, and assessing learning outcomes. In contrast, the traditional academic quality model focuses mainly on content: what should be taught, not how it should be taught and learned and how learning should be assessed. Largely missing from the traditional model are these elements of EQW:

• quality defined as fitness for use as determined for the kinds of students served by the institution;

• intensive focus on the processes of teaching and learning;

• systematic tradeoffs among of faculty time and other resources spent on educational tasks;

• vigorous performance assessment of students, teachers, and the systems in which they operate;

• decisions based mainly on facts, informed but not dominated by the standards of the academic disciplines; and

• well-organized processes for continuous improvement based on feedback.

Faculty and institution-level EQW support and stimulate departmental and program level work. Providing leadership, resources, incentives, information, training, and interdepartmental venues for discourse on quality work provide examples of school and institution-wide activities. Such activities also should include periodic evalualion of work at the departmental level in order to ensure internal accountability and provide impetus for improvement.

Quality oversight, of which the TLQPR is an example, energizes institutional EQW and assures its effectiveness. Review by agencies like the UGC can establish the quality work agenda and stimulate discourse as well as monitor progress. By evaluating quality work rather than trying to assess the delivered quality of education, oversight bodies can discharge their accountability obligations without resorting to disruptive assessment practices or intrusive regulation. To use the Swedish terminology once again, the external entity should use a Iight touch" to stimulate improvement while discharging its public responsibilities.

NCPI interviews in Sweden, Denmark, UK, Netherlands, Australia, and the U.S., and our own experience here in Hong Kong, have convinced me that there is no inherent conflict between the improvement and accountability agendas when EQW oversight is performed correctly. Nor is there any reason why such oversight can't be "owned" by the entity responsible for funding. (The actual reviews should be performed in close consultation with the institutions, perhaps with the assistance of third parties.)

There also is no reason why the reviews should not "inform funding", although not in any formulaic way. Indeed, the funding agency would not be doing its job if it did not use all available information, including information about the state of educational quality improvement and assurance, as part of its subjective evaluation of institutional performance,

Diffusion of EQW won't take place overnight. It may take years to refine the concepts and methods, build expertise, and embed EQW principles firmly in the academic culture. People who seek a quick fix will be disappointed. The adoption of innovations usually starts slowly. It does not really accelerate until a critical mass of successful experience has been achieved. We should not expect the adoption of EQW to behave differently.

Viewing the adoption of EQW as a diffusion process underscores the need for an ongoing TLQPR program. As the CHEPS team has told us, it is important to keep up the beat until EQW becomes deeply embedded at both the university and departmental levels. Some kind of external review should be maintained even then. Halting the program at any stage would send the message that EQW is no longer important and discredit EQW advocates within the universities. As soon as this happens, other priorities such as research will erode EQW's priority. The reviews can be operated in conjunction with other quality assurance exercises and with a lighter touch once universities have demonstrated their own ability to perform quality oversight, but the need for some kind of external oversight program will continue indefinitely.

Once again, thank you for helping the UGC prepare for the next round of TLQPRs. My colleagues and I are most interested in your views, and we look forward to a stimulating day.