Authors
Dr.
Agnes Tiwari, Assistant Professor, Department of Nursing Studies, Faculty of
Medicine, The university of Hong Kong, G/F., Block B, Queen Mary Hospital,
Pokfulam Road, Hong Kong.
Dr. Catherine Tang, Head,
Centre for Learning, Teaching and Supervision, Hong Kong Institute of Education
Title
Does
portfolio assessment encourage students to adopt different assessment
preparation strategies?
Number of words
1947
Abstract
Studies
have shown that students select different preparatory techniques for different
assessment tasks. For example, if the
assessment task is perceived as the reproduction of previously learned content,
rote learning would be adopted as the preparatory strategy. The perceived demand of an assessment,
therefore, exerts a strong influence on the assessment preparation strategy
adopted. Portfolio assessment is
designed to promote a deep approach to learning. However, empirical data are lacking to demonstrate the effect of
portfolio assessment on students' approaches to learning. The purpose of this study is to examine the
impact of portfolio assessment on students' study strategies in preparation for
their assessment, using the Assessment Preparation Strategy Questionnaire
(APSQ).
Key words
Portfolio
assessment, study strategies for assessment, Assessment Preparation Strategy
Questionnaire (APSQ).
Text
Students’ learning is to a large extent affected by assessment tasks (Biggs, 1999). If the assessment task is perceived as a reproduction of previously learned content, students are more likely to adopt a surface approach to learning. On the other hand, if students perceive the assessment task as achieving personal understandings of the content, a deep approach to learning is more likely to be adopted. Empirical evidence exists to support the suggestion that different formats of assessment produce different approaches to learning (Scouller, 1998; Tang, 1994). As a form of assessment, portfolios are designed to encourage students to adopt a deep approach to learning. Indeed, the students in Scouller’s study see portfolios as ‘a powerful learning tool’ (P. 136). Despite the promises, there is relatively little research on the effect of portfolio assessment on students’ approaches to learning. Therefore, one of the objectives for undertaking this study is to examine the actual study strategies employed by the students in preparation for their portfolio assessment, using the Assessment Preparation Strategy Questionnaire (APSQ) (Tang, 1991).
Portfolio Assessment
Portfolio assessment is defined in this paper as a collection of student’s work, purposefully collected and selected, as a means of documenting the process of learning and demonstrating student’s achievement. Portfolio assessment stems from a Constructivist theory of knowledge (Biggs & Tang, 1998) and is based on the premise that meaning cannot be imposed or transmitted by direct teaching but created by the students through their learning activities. Since meaning is constructed, students should therefore provide evidence from their constructions of knowledge to show that the desired learning has occurred.
Portfolio assessment is comprised of three processes: (1) setting the criteria for assessing the work; (2) selecting the evidence that would be relevant to judgment against those criteria; and (3) making a judgment about the extent to which the criteria have been met (Biggs, 1999: 157). While the teacher may still set the criteria for assessing as in conventional assessment, the student is aware of what is required as evidence of learning in portfolio assessment; and s/he takes over from the teacher in deciding what evidence of learning is to be submitted. Thus, the key difference between portfolio assessment and conventional assessment is that the student decides what should be included in the portfolio as evidence of his/her learning. In deciding what constitutes good learning, the student is aware of the course objectives and the benchmark against which his/her work will be judged. The student then selects examples of his/her work that best match the course objectives and the benchmark. The portfolio assessment in this study requires each of the students to submit two pieces of work for the portfolio.
Methods
The nonequivalent control group design (Cook & Campbell, 1979) was used for this study. Seventy (70) nursing students in the Department of Nursing Studies of the University of Hong Kong were involved. Twenty-one (21) of these students were enrolled in the part-time post-registration Bachelor of Nursing programme while forty-nine (49) were in the full-time pre-registration Bachelor of Nursing programme. The part-time students were registered nurses and the full-time students were working towards their registration. The nonequivalent nature of the two groups was recognized, however, due to curricular constraints, it was not possible to implement an experimental design. In view of the weakness, caution was exercised when analyzing the results. Both groups undertook a one-semester course in “Nursing Theory”; portfolio assessment was used as the assessment strategy for the part-time students (‘portfolio’ students) while the full-time students remained on the conventional assessment (‘conventional’ students). For the portfolio assessment, students had to submit two items of their work together with a statement of justification for the work to be included in the portfolio. Students in the conventional assessment had to submit one essay assignment and take an end-of-term test. Approval was obtained from the Faculty Ethics Committee prior to the study. The students who participated in the study gave their written consent after a thorough explanation of the study and their rights as research subjects. The study was undertaken to evaluate the impact of portfolio assessment on student learning, in this paper only the part that focuses on assessment preparation strategies will be reported, that is, the actual study strategies employed by the students in preparation for their assessment. In order to measure the students’ assessment preparation strategies, the APSQ was administered at the end of the assessment procedure to both groups of students. In addition, the students were interviewed at the end of the course and asked to elaborate on their study strategies in preparation for the assessment.
The Assessment Preparation
Strategy Questionnaire (APSQ)
The Assessment Preparation
Strategy Questionnaire (APSQ) was first designed and implemented by one of the
co-authors of this paper in a study of the effects of different assessment
methods on students’ approaches to studying in a physiotherapy education
programme (Tang 1991).
Based on the surface, deep
and achievement study approaches items of the Study Process Questionnaire (SPQ)
developed by Biggs (1979), the APSQ items were designed to explore the
strategies which the students employed when they were preparing for two
different assessment methods, written test and assignment. The questionnaire
consisted of 30 items constituting 5 factors:
Factor
1 – Learning materials and processing.
This factor examines the sources of study materials and the processing of the information
Factor 2 - Narrow focusing and cue seeking
This factor describes the focus of studying and whether or not the students are consciously aware of and seek cues.
Factor 3 - Professional perspectives
This factor describes whether or not the students relate what they are studying to their professional context.
Factor 4 - Academic perspectives
This factor describes whether or not the students relate what they are studying to other subjects.
Factor 5 - Achieving strategy
This factor describes the students’ organization of work and study time.
In the current study, the APSQ used was adapted from the original APS questionnaire to explore the actual study strategies employed by the students in preparing for their portfolio assessment. The questionnaire consisted of 29 items describing the 6 factors:
Processing information; Focus of learning; Professional perspective; Academic perspective; Achieving strategy; and Affective and perception of effects (Detail explanation of the 6 factors and sample items are included in Appendix A).
Results
With reference to the APSQ, the ‘portfolio’ students scored highest on factor 4 (27.31) , second highest on factor 5 (18.81), and lowest on factor 3 (8.25). The same scoring pattern was also found among the ‘conventional’ students. When the two groups of students were compared, significant differences were noted in factor 3 (t =
-4.14, p = .000) and factor 4 (t = -2.182, p = .033), with the ‘portfolio’ students scoring higher in both of the factors. No significant differences were observed in the other factors.
The interview data converged with the APSQ findings. Both groups of students stated that they were relating what they were studying to the other subjects. They also described themselves as organized in their organization of work and study time. Both groups of students did not view what they were studying as practical enough for application to clinical practice; however, they gave different reasons for this.
Discussion
Among the six factors of the APSQ, both groups scored highest on factor 4, thus suggesting that in relating what they were studying to the other subjects, they were adopting a deep approach to learning. For the ‘portfolio’ students, this may suggest that the portfolio assessment has achieved its desired effect, that is, the students were taking a deep approach to learning by trying to understand what they were studying through the process of relating it to the other subjects. But why should the ‘conventional’ students score high on this factor as well? An examination of the assessment procedure for the ‘conventional’ students has revealed that there was an assignment component. In principle, assignment allows for deep learning as the students can consult more sources and therefore synthesize more effectively with the deeper knowledge base (Biggs, 1999). Thus, the high score observed in the ‘conventional’ students could be the direct effect of the assignment component in their assessment procedure. Both groups of students talked about how they related what they were studying to the other subjects in the interviews:
Before, I would only go to the nursing subject to find nursing materials. Now, I realize that it doesn’t have to be like that. Indeed, I can look up other sources, such as philosophy. I think it helps me to be creative by not restricting me. Doing homework can be fun, and not boring at all. (‘Portfolio’ student)
The assignment on Orem’s theory will always remind me of the need to search broadly. It is not just a theory issue, not just a nursing issue; it is about how a patient may or may not accept a certain way of caring. So, I had to look beyond the subject of nursing theory. (‘Conventional’ student)
Factor 5 or the students’ organization of work and study time was the second highest score by both groups of students. The importance of being organized was evident in the interview data:
We got together to discuss what we had learned, just the three of us, with our babies as well. We were feeding the babies while doing the discussion. With so many demands on our time, we had to be organized. (‘Portfolio’ student)
There was so much to do, attending lectures, finding information, doing practicum, and preparing for assignment. I had to decide how much I would do for each of these. The practicum took up a lot of my time and we had no choice in this. (‘Conventional’ student)
It would appear that although both groups of students saw being organized as important for their study, they did so for different reasons. The ‘portfolio’ students, being part-time students, had competing demands made on their time, as employees, students, and carers for their family. For the ‘conventional’ students, even though they were pursuing a full-time study, they felt the need to organize themselves due to the packed curriculum.
Interestingly, both groups of students were less inclined to relate what they were studying to the patients’ conditions in clinical practice, as evidenced by the students scoring the lowest score in factor 3, compared with the other factors. In the interview, the ‘portfolio’ students and ‘conventional’ students gave different reasons for this. The ‘portfolio’ students viewed the subject “nursing theory” as not relevant for the real-life situation even though they liked the idea of doing portfolio assessment:
The portfolios allow you to look up things that can be used in practice…but nursing theory is too theoretical. I sometimes wonder whether the objectives can be achieved. Adopting the theory in practice is quite another matter.
The ‘conventional’ students were skeptical about the application of the subject to the clinical situation because they lacked the experience of putting it to practice:
I remember writing the care plan based on what I had learned. Although I tried to apply the plan of care for over two weeks, I had no idea whether it was workable. I wrote that the old lady should do this and that but in practice, did she do it?
Conclusion
Portfolio assessment as applied in this study appears to have made some impact on the students’ study strategies in preparation for assessment. This is particularly evident in how they related what they were studying to the other subjects and in being organized in their work and study time, as shown in their APSQ scores. The findings, however, should be viewed in the context that students undertaking the conventional assessment also had high scores in those factors.
References
Biggs, J. (1979). Individual difference in study processes and
the quality of learning outcomes. Higher
Education, 8, 381-394.
Biggs, J. (1999). Teaching for quality learning at university. Buckingham: SRHE and Open University Press.
Biggs, J., & Tang, C. (1998). Assessment by portfolio: Constructing learning and designing teaching. In P. Stimpson & P. Morris (Eds.). Curriculum and assessment for Hong Kong – two components, one system (pp.443-462). Hong Kong: Open University of Hong Kong Press.
Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally.
Scouller, K. (1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination vs. essay assignment. Higher Education, 35, 453-472.
Tang, K.C.C. (1991). Effects of different assessment procedures on tertiary students’ approaches to studying. Unpublished PhD Thesis. The University of Hong Kong.
Tang, C. (1994). Effects of modes of assessment on students’ preparation strategies. In G. Gibbs (Ed.). Improving student learning: Theory and practice. P. 151-170. The Oxford Centre for Staff Development.
This factor examines the processing of the information
APSP1: Memorizing and summarizing
Sample item: I summarized the
information from the reference materials and then put it in the portfolio.
This factor describes the focus of studying and whether or not the students are consciously aware of and seek cues
APSP2: Narrow focusing and cue seeking
which the lecturers would expect us to include in the portfolio
This factor describes whether or not the students relate what they are studying to patients’ conditions in clinical practice
APSP3: Deep-professional
Sample item: When I was preparing the portfolio, I put
myself in the position of the nurse and tried to understand his/her role in the
management of the patients.
This factor describes whether or not the students understand and relate what they are studying to other subjects
APSP4: Deep-academic
Sample item: While I was preparing the portfolio, I
found that I could relate this subject’s information to the other subject
areas.
This factor describes the
students’ organization of work and study time
APSP5: Organizing
Sample item: I studied the subject materials regularly
after the tutorials with the view of preparing for the portfolio.
This factor examines the students’ feeling towards and their perception of the effects of the assessment method
APSP6: Positive
Sample item: I was confident about the portfolio as I
knew that I had prepared it well.