Exit test

Exit tests have been suggested as a way to ensure a certain language proficiency level in university graduates as well as a mode to foster greater accountability on the part of language enhancement providers. Participants in several workshops deliberated on the issues surrounding exit tests. Several reservations were expressed. They included:

• One standardized test alone is unhelpful in measuring the immense variation across programmes, disciplines and institutions. What the test should focus on is debatable, even if restricted to the broad areas of General Proficiency, Language for Academic Purposes and Language for Professional Purposes.

• Resources must be found to prepare students for the test, if it is implemented.

• For valid comparison with the exit test, there must also be a relevant entry test.

• Given the increase in student intake and the inevitable lowering of entry levels on the average, it is not realistic to set high exit levels across the board unless there is much more time and learning resources for language enhancement.

• The general conclusion is that guidelines, rather than a standardized test,will be more feasible.

There was general agreement among participants that input from non-language teachers is valuable in at least two ways. Non-language teachers are important resource persons for the development of language learning materials because language enhancement cannot be fostered in a knowledge vacuum. Ideally, relevant content areas have to be built into the language programme. Secondly, if language fluency and accuracy are more emphasized in the classes and assignments for non-language subjects, then students will be more motivated to enhance their language competence. This is especially important in view of the fact that most language enhancement programmes are taken early on in the student's academic life.

Level of Chinese proficiency

Measures taken to alleviate the situation

A required standard in Chinese proficiency before graduation

Participants agreed that it would be extremely difficult to have a common required standard. Differences in the learning environment and personnel also meant that the requirements would be different. But, it was still possible to try to achieve the following objectives:

• Teachers should place special emphasis on the students' acquisition of basic language skills. Once a student had mastered the basic skills, he/she would find it easy to learn practical Chinese or Chinese for specific or professional purposes.

• Each institution should find ways to encourage students to become independent and self-directed in their learning. Some participants pointed out that second and third year students very often had a lower level of Chinese proficiency than the first year students. This is because in their institutions most of the language enhancement is provided in the first year. This indicates clearly the importance of motivating students to learn by themselves.

• All agreed that the Chinese programme in all the institutions needed more teachers, who could then improve the quality of teaching. If the above objectives were achieved, the participants were confident that our tertiary students would have a higher level of Chinese proficiency.

Entry and exit levels of proficiency

Colleagues from different tertiary institutions reported on the minimum grade required for admission. In general, most institutions enforced this requirement strictly, allowing in only a few students who scored high in their major disciplines and low in language proficiency. There was, however, the feeling, among some colleagues, that the grade obtained in the Use of English might not be a very reliable indicator for successful studies at university level and by imposing a pre-set standard, some very good students might be sacrificed.

Appropriateness of an exit test

There appeared to be reservations among the participants for a uniform exit test across institutions because of the different requirements in different disciplines. Some colleagues observed that if the exit test was business-oriented, then students not taught the skills would score low even though their English proficiency was high. Other colleagues were concerned about the timing of the test as enhancement courses were given in the first year and the exit test might be administered in the final year.

Status of courses and student motivation

Colleagues from different institutions shared with the group the kind of English enhancement programmes they had. They also discussed the following dilemma: requiring only students scoring below C in the HKCEE to take courses might put a stigma on the low achievers, while requiring all first years to attend English classes might leave inadequate funding for English programmes in the second and third years.

Colleagues pointed out that if the ability to express ideas in English was counted towards the overall grade in assignments in other disciplines, then students would pay more attention to English. Colleagues at the Chinese University and the City University reported that they were at the initial stages of experimenting with Writing Across the Curriculum where faculty members from other disciplines would be invited to work with English language teachers to help students to write better.

Entry levels of Putonghua proficiency

Exit levels of Putonghua proficiency and appropriateness of an exit test

Credit-bearing status of courses and student motivation

Enhancement beyond the classroom

Curricular emphasis

Difficulties

Purpose of the funding

Some of the participants pointed out that the aims of the enhancement funds had not been explicitly defined by the UGC. They were not sure whether the funds were meant for remedial purposes or for enhancement in language proficiency for all tertiary students. If the funding was for remedial purposes, it would only cater to the needs of those students whose level of proficiency was comparatively weak and would exclude those who were competent. A participant pointed out that even those who received a high grade in Chinese in the HKAL would not necessarily be able to write a business letter properly.

On the other hand, if the funding was for further language enhancement, the outcomes would be different. It would help not only those who are weak in Chinese but also strengthen the proficiency of those with good academic results - for example, a Grade A or B in Chinese Language in HKAL. At present, this group of students are excluded from some of the enhancement programmes. By extending the coverage of the programmes, the resulting benefits would be two-fold. Firstly, the students could enrol in one or more of the courses in the enhancement programme so that they could strengthen their language proficiency to meet academic requirements. Secondly, the high level of proficiency acquired through the courses would enable them to speak and write in Chinese with confidence in their careers upon graduation.

Timing of funding allocation

Under the present allocation policy, funds are allocated on a triennium basis by the UGC. But the actual allocation is reviewed by the administration in each institution annually. This arrangement has given rise to three problems:

• A participant mentioned that his department was only able to offer six-month contracts to some of the teaching personnel. It was because there was uncertainty about how much funds would be allocated to the department the following year. As a result, it was difficult for the department to formulate its long-term strategic plan.

• Another participant reported that some of the teaching staff in his department were employed on a contract basis. Their contracts would expire in June but the number of establishments for the following academic year would not be confirmed until a much later date. As a result, some of the contract staff might seek other job opportunities outside the institution. Moreover, owing to an uncertain future, they might have less incentive to diligently prepare for or improve teaching materials. To a certain extent, this might hinder the process of developing courses.

• The above two problems eventually led to a third problem. It is difficult to employ highly qualified teaching staff because of the uncertain future. As a result, it may eventually make it difficult to achieve the objective of language enhancement - to improve the language proficiency of students.

Basis of allocation

Exit test

The providers of language enhancement programmes are accountable to providers of funds (the UGC and the institutions) and responsible to participants in programmes supported by the funds for the learning quality of the language programmes. An exit test has been proposed to provide a yardstick by which the English competence of graduates can be judged. The performance of students on a standardized exit test might also give some indication of the effectiveness of programmes. At the workshop, however, several problems concerning a standardized exit test were raised:

• With an intake of 18% of the relevant age group into tertiary education, it is inevitable that standards of entrants will not be all high. It does not seem feasible then to set high exit standards.

• Different disciplines might also attract students with different proficiency levels to begin with and it might therefore be unrealistic for some disciplines to require a high exit standard of English.

• In some institutions, the English courses are optional and tend to be taken by students whose English is already rather good to improve their Grade Point Average (GPA). In these circumstances, it does not make much sense to have them pass an exit test.

• If there is an exit test, there will need to be a pre-test for valid comparison as well as teaching resources to help students prepare for it.

• If the test focusses on competence in General English, it will have overlooked the fact that even students with good General English still need enhancement in Academic or Workplace English.

At present, institutions have dealt with exit standards differently:

• Some provide a Pass/Fail for the English course on the transcript with no further discrimination into letter grades.

• Some require the students to take one or more course(s) and pass the achievement tests at the end of the course(s).

• Some make no specific requirements. Courses are optional.

Alternative activities for funding support

What students need might not be just more courses but also enhanced environments for the use of English. Tutorial systems, self-access centres (provided the students use them) are possibilities. The role of non-language teachers is also very important. Some institutions have tried to provide some enhancement for teaching staff but this has to be approached with sensitivity. It is important not to make teachers (language or non-language ones) the scapegoats of language competence levels as a result of historical circumstances. The culture surrounding the learning of English in Hong Kong needs reconsideration. In some European universities, for example, while competence in the first language is an objective, competence in English as an international language is still important. International student exchange programmes might help in opening up environments for our students. Exchange programmes require curriculum time, which might be more available in a four-year curriculum. Finally, employers also have a part to play in offering opportunities for workplace experience to students.

Specific to the funding of enhancement programmes were two requests from participants:

1. In the midst of funding cuts to the block grants, enhancement programmes, which usually receive some topping-up from the block grants, are not spared. This is unfortunate because the improvement of language skills demands small group interaction. With funding cuts, increased class sizes will diminish opportunities for individual class participation.

2. Since language learning is a very complex phenomenon, it is better not to relate funding to exit standards or even entry standards of students.

Materials in current use

Participants gave a brief account of the teaching materials of Chinese and Putonghua which are currently used. All the institutions use their own materials.

The Department of Chinese at CUHK has developed successively many kinds of materials. Some were published. Some are outdated and no longer used. These packages include materials on classical and modern Chinese, practical writing and a basic knowledge of Chinese. The materials on Putonghua were compiled by the Language Self-Study Centre and Lau Ming.

The Department of Chinese at HKU developed materials for supplementing the students' knowledge of Chinese. Materials are so designed that students may have a good command of practical Chinese and develop interest in the Chinese language and culture. Courses and materials vary according to the demands of different departments. Practical Chinese writing and Putonghua form the common core in the teaching programme. Simplified characters, grammar and rhetoric, speaking skills and Chinese for academic use are also included.

HKIEd keeps its focus on training teachers for primary and secondary schools and this focus is reflected in their materials, which are separately prepared for secondary and primary sections. Chinese teaching methodology and a basic knowledge of Chinese are covered in their materials. The materials on Putonghua and Chinese are not yet published.

HKUST will offer courses in practical writing. The materials on Putonghua are compiled for students at elementary, intermediate and advanced levels. Besides, some Putonghua courses are designed for business communication. No materials on Putonghua and Chinese are published as yet.

No information is available on LC's Chinese teaching materials since there was no representative from the Chinese Department at the workshop. The Putonghua teacher present at the seminar reported that their Putonghua courses were classified as credit and non-credit ones but the materials were not yet published.

CITYU did not have any Chinese teacher present, so no details concerning their Chinese programme were obtained. The Putonghua teacher present explained that there were four divisions to offer Putonghua courses at the university. All divisions developed materials for their own use.

The Language Centre of HKBU provides Chinese courses for many departments and its text materials are of two kinds: for enhancing students' writing abilities and for training their basic language abilities, including the reading and writing of simplified characters, practical writing and so on. The textbooks on Putonghua are compiled by their teaching staff.

Sharing and collaboration in materials development

As a way to improve the current situation of compiling teaching materials only for internal use in each institution, some participants proposed inter-institutional collaboration for the tasks listed below. Applications for UGC funding could be made:

1. Revising the current materials and publishing them for exchange and reference so that all may learn from the strengths of others to offset their own deficiencies.

2. Developing new materials jointly so as to save time and labour.

3. Putting the materials onto the computer network for public use and reference.

Other matters

Materials for use in specific departments

The discussion on the development of materials written to meet the needs of students within specific departments revealed that the degree of collaboration between subject and language teachers varied considerably both within institutions and between institutions. The difficulty of developing materials for English for Academic Purposes (EAP) and ESP whose content is acceptable to both students and faculty members highlighted the need for consultation between language and subject teachers. Although there was a general willingness to share materials with other institutions, it was felt that this was not always practical because not only were courses always evolving but the materials were usually too specific to both course and students to find a wider audience.

There was considerable support for more collaboration between subject and language teachers. It was suggested that language teachers could be assigned to subject teachers but it was generally agreed that the outcomes were likely to be more positive when teachers established their own support network. One institution reported offering staff in-service language enhancement courses. These courses were well received and although such courses may appear remote from materials development, they may serve to build up goodwill between language teachers and faculty members that will lead in turn to the greater collaboration that is seen as so desirable. It was suggested that HK tertiary institutions should establish a register of language and subject teachers interested in collaborating on materials development.

Materials for Self-Access Centres

The development of materials for self-access centres was dealt with quite briefly because of the time constraint. Reference was made to the Hong Kong Association for Self-Access Language Development (HASALD) whose members meet regularly and are willing to exchange materials. It was noted that materials developed for self-access use, like those developed for students in different departments and faculties, may not always transfer successfully from one institution to another. Furthermore, the need for self-access materials to be accompanied by appropriate learner support was stressed. It was suggested that a data bank of materials could be put on the web. One participant reported on the difficulty of building up a computer data base of materials even within her department and the problem of copyright was also raised. However, the idea of materials being available on the web was recognized as promising, especially if the bank included locally based materials.

What is being done

1. Chinese enhancement programmes: On the whole, the institutions have been offering a wide variety of Chinese programmes. Beyond regular courses, the institutions have offered voluntary/elective courses (on topics such as practical writing, oral communication, Chinese computer skills), self-study programmes (such as reporting selected literary works), writing clinics and so on. Each institution has a different emphasis in their programme, depending on the nature and history of the institution, the needs of the students and so on.

2. Methods of assessment: The following methods for assessing students' performances were reported: on-going assignments, tests, final examinations, keeping attendance records and teacher's reports. In one university, students were awarded for good performance with a Certificate of Merit based on the teacher's report and documentary support. Self-assessment through feedback from the computer was also introduced. The participants also noted that in one institution the departments had some resistance against assessment in the form of a final examination/test.

3. Programme evaluation: As for the evaluation of programmes, one institution reported using student questionnaires. Students commented on the structure of the course, its appropriateness and effectiveness, the performance of the teacher and so forth.

What has caused concern

1. Wide range of student ability: The wide range of ability in a course could lead to difficulties in content design, materials selection and the setting of assignments. The allocation of grades would also be difficult especially for students at either end of the ability range. Low grades could dampen students' morale. This difficulty was more strongly felt with voluntary/elective courses. The method of screening and streaming students according to ability was raised as one possible solution though it might not apply to all situations.

2. Comparability and standardization: It is difficult to have standardization in the awarding of grades across teachers, courses, departments and even institutions.

3. Suggested solutions

   The following methods were reported as partial solutions:

   1. setting a significant portion of standardized/objective component in assessment (e.g. 50% each for the standardized and non-standardized parts);

   2. using normal distribution to regulate grade allocation;

   3. an external examiner monitoring and regulating the standards;

   4. markers' meetings (including given criteria, marking sample scripts and moderation by coordinator);

   5. standardized exit test.

   Exit test

   It was felt that, on the one hand, an exit test could provide a standard for reference within an institution and, perhaps, even across institutions. On the other hand, concern was raised about the ensuing difficulties, e.g. effects on the autonomy of institutions, effects on flexibility in teaching and the washback effects on teaching/learning. It was proposed that the UGC could play a direct role in settling this matter.

   Sharing materials

   It was also pointed out that institutions can share materials for learning and assessment, including question banks. This can better align the institutions in their teaching, diagnostic work and assessment. It was again proposed that the UGC could play a stronger role in promoting this.

4. Assessment of self-access learning: Several institutions have yet to have their own Chinese self-access centres established. Experience in this area is still lacking though one institution mentioned that they already had self-study programmes using the computer. Students can assess their difficulties by doing exercises on the computer. They can check their answers and a tutor can provide help.

Assessment of English

The first area, although broad, was only lightly touched on. There was discussion of whether or not students who were weak in English, but good at their content subject, should be allowed to continue their studies at university. Several participants disapproved of the policy of barring otherwise talented students on the basis of their English skills, but it was pointed out that English standards had to be maintained, and, just as importantly, seen to be maintained.

Exit test of English proficiency

A particular issue in the discussion was the exit test being developed at Polytechnic University. It was felt that we, as English teachers and course-writers, are obliged to provide some sort of evaluation in order to meet the requests of employers.

However, several concerns were expressed about the concept of an exit test. The two main ones related to:

1. whether a single test can be appropriate for students receiving a variety of English instruction, taking a variety of subject courses, at a variety of institutions;

2. the idea of a final exit test being 'added on' to existing course-integrated evaluation.

Suggestions/comments that were made included the following:

1. assessment should be ongoing and integrated into courses; what was important was curriculum reform to produce needs-specific courses and certification (rather than a uniform standardized test);

2. a better alternative might be the Hong Kong Vocational English Programme, designed for those already at work and comprising courses and integrated skills assessments (i.e. the skills being tested are ones that the students have had real experience of);

3. it is not just language/communication skills that are important for success in the workplace; social skills should also be developed and assessed.

Evaluation of self-access learning