(PDF) And Teachers ’ Criteria
Academia.edu uses cookies to personalize content, tailor ads and improve the user experience.
By using our site, you agree to our collection of information through the use of cookies.
To learn more, view our
Privacy Policy.
About
Press
Papers
We're Hiring!
Outline
Title
Abstract
Literature Review
Method Participants
Instruments and Data Collection Procedures
Data Analysis
Results
Discussion
Conclusion
References
And Teachers ’ Criteria
Roya Pashmforoosh
2016
visibility
description
27 pages
Sign up for access to the world's latest research
check
Get notified about relevant papers
check
Save papers to use in your research
check
Join the discussion with peers
check
Track your impact
Abstract
Perceptual (mis)matches between teachers and learners are said to affect learning success or failure. Self-assessment, as a formative assessment tool, may, inter alia, be considered a means to minimize such mismatches. Therefore, the present study investigated the extent to which learners' assessment of their own speaking performance, before and after their being provided with a list of agreed-upon scoring criteria followed by a practice session, matches that of their teachers. In so doing, 29 EFL learners and six EFL teachers served as participants; the learners were asked to assess their audio-recorded speaking performance before and after their being provided with the scoring criteria and practice session. The teachers were also asked to assess the learners' performance according to the same criteria. Finally, the learners were required to evaluate the effectiveness of doing self-assessment in the form of reflection papers. The results revealed a significant difference between the learners' assessment of their own speaking ability on the two occasions. The findings also suggested that providing the learners with the scoring criteria and the follow-up practice session minimized the existing mismatches between learner assessment and teacher assessment. Moreover, the inductive analysis of the reflection papers yielded a number of themes suggesting that, despite some limitations, the learners' overall evaluation of the effectiveness of speaking self-assessment was positive.
Related papers
Exploring The Assessment In Speaking Classroom
nurul husna
2020
In speaking assessment, the lecturers pay more attention of how to develop students’ speaking skill rather than assessing the speaking itself. Moreover, EFL students usually face some challenges in speaking classroom. The aims of this research are to investigate kind of assessment used by lecturers in the speaking classroom and to explore the EFL students’ challenges towards the assessment in speaking classroom. This research was conducted in Fakultas Tarbiyah dan Keguruan Department of English Language Education. The subject of the research were 5 English teachers who have been teaching speaking for five years and 100 students in the seventh and ninth semester at academic year 2016 and 2015 who had taken Public Speaking and English Language Assessment course. This research implemented mix method by using interview and questionnaire and it was analyzed descriptively. The result of this research revealed that: all lecturers’ put their objectives in assessing students’ speaking skills...
Download free PDF
View PDF
chevron_right
Developing Self Assessment Instruments for Teaching Speaking to Grade Eight Students of Junior High School
asep ardiansyah
The main purpose of this study was to develop self assessment instruments for speaking competency of the eighth grade students of junior high school. The focus of this study was specifically on; 1) finding out the types of self-assessment needed to be developed; 2) developing the prototype of self assessment instruments; and 3) examining the quality of the developed selfassessment instruments. This study used R&D method of Gall, Gall & Borg model. The subjects of this study were eleven English teachers and syllabus for the eighth grade students of junior high school. The data were obtained through questionnaire, interview, observation and syllabus analysis. The results revealed that there were two form of selfassessment instruments needed to be developed, namely linguistic aspect and non-linguistic aspect self assessment. The prototype of self assessment instruments were developed by considering the need analysis results and grand theory. The instruments were validated and revised by two expert judges and user judges. Based on the validation from expert judges and user judges, the quality of the developed self assessment instruments was categorized as excellent instruments with very high validity.
Download free PDF
View PDF
chevron_right
Assessing speaking skills: a workshop for teacher development Ben Knight Downloaded from
Ben Knight
Speaking skills are often considered the most important part of an EFL course, and yet the difficulties in testing oral skills frequently lead teachers into using inadequate oral tests or even not testing speaking skills at all. This article describes a workshop used in teacher development programmes to help teachers with one aspect of the problem of oral testing: what should we look for when we assess a student's ability to speak English? The workshop looks first at the range of criteria that teachers might use in such assessment. Then it examines how the selection and weighting of those criteria should depend on the circumstances in which the test takes place. The article also discusses issues raised by the workshop, and considers its applicability to people working in different circumstances. Reasons for the Assessment of speaking skills often lags far behind the importance given workshop to teaching those skills in the curriculum. We recognize the importance of relevant and reliable assessment for providing vital information to the students and teachers about the progress made and the work to be done. We also recognize the importance of backwash (the effect of the test on the teaching and learning during the course). Most teachers would accept that 'if you want to encourage oral ability, then test oral ability' (Hughes, 1989:44). But the problems of testing oral ability make teachers either reluctant to take it on or lacking in any confidence in the validity of their assessments. Such problems include: the practical problem of finding the time, the facilities and the personnel for testing oral ability; the problem of designing productive and relevant speaking tasks; and the problem of being consistent (on different occasions, with different testees and between different assessors). Another problem, which is the focus of the workshop framework described here, is deciding which criteria to use in making an assessment. The workshop has two principal aims: 1 to make teachers more consciously aware of the different possible criteria they could be using to assess their students' speaking skills; 2 to make teachers more aware of the way their selection and weighting of those criteria depend on the context in which they are to be used. Achieving these aims is crucial for making valid and reliable tests. Except where tests are being marked holistically (simply in terms of degrees of communicative success), marking involves the use of assessment criteria. Even when the assessment is holistic on the surface, the assessor may be thinking in terms of criteria in judging that overall communicative success (Bachman, 1990: 329). It is doubtful whether the criteria can be
Download free PDF
View PDF
chevron_right
Teachers' assessments and students' self-assessments: do they correspond
Science Park Research Organization & Counselling
Alternative methods of assessment have recently attracted the researchers' and educators' attention. In this sense, self-assessment as a significant learning strategy for autonomous language learning has been the focus of much research. The current study attempted to investigate whether Iranian EFL students' self-assessments would correspond to their teachers' assessments considering the productive skills. It also aimed to see if the teachers' experience would influence the correspondence between the two assessments. Hence, 100 beginner EFL students (in six separate classes) were asked to self-assess their own writing and speaking skills. Their corresponding teachers (six ones) were also asked to assess their students. The results of descriptive statistics and a series of independent t-tests showed that students' self-assessments were in accordance with their teachers' assessments and the teachers' experience would not predict the correspondence between these two evaluation scores. According to the results, it is recommended that teachers employ self-assessment techniques in their classes to involve their students in the process of evaluation. Some suggestions for further research are also presented.
Download free PDF
View PDF
chevron_right
Designing Criteria to Assess Speaking Skills
Stanislav Rychtařík
This graduate thesis deals with designing criteria to assess speaking skills. The first section, serving as a theoretical basis for the subsequent research, presents readers with the reasons for selecting this topic and explains the terms speaking, assessment and assessment criteria in the context of assessing speaking skills. The research, conducted by means of teacher questionnaires, maps the teachers' current practices in terms of assessing speaking skills. The results of the research revealed the most frequently employed speaking assessment tasks, teachers' preferences in types of assessment and the fact that the criteria teachers select for individual speaking assessment tasks differ quite significantly. Subsequently, the results are commented on, and possible reasons for their occurrence are mentioned. Implications of the results for language teaching are discussed at the end of the thesis.
Download free PDF
View PDF
chevron_right
Developing Assessment for Speaking
Yenny Rahmawati
IJEE (Indonesian Journal of English Education), 2015
Recently there have been debates on assessing students’ performances on speaking since the cultural and subjective issues embedded in bringing awareness on how teachers construct their speaking assessment. The main focus of this paper is a way to design assessment for speaking suitable for the Indonesian context at a university level. This paper stresses the criteria of effective assessment proposed by Brown and Abeywicrama which consists of a specific criterion, an appropriate task, a maximum output and practical and a reliable scoring procedure. It is recommended that teachers develop their speaking assessment which is appropriate and contextual.
Download free PDF
View PDF
chevron_right
On the comparative impact of self-assessment and peer assessment on Iranian male and female EFL learners’ accuracy in speech
Atefeh Ferdosipour
Contemporary Educational Researches Journal
The current study tried to explore the effects of assessment types and learners’ gender on EFL learners’ performance in speech accuracy. To this end, 60 homogenous students out of 90 students from Rahe Farda Institute in Bastak in Hormozgan province were selected based on the Quick Oxford Placement Test. The instruments of the study were an Oxford Placement Test, peer and self-assessment questionnaires adopted from Patri and Weir’s Analytic Speaking Criteria. The results were calculated using SPSS version 20. It can be seen that the means were close together, but the mean score for the male students was a little higher than that of the female students. According to the outcomes, assessment types had a significant influence on performance in speech accuracy and peer-assessment was more effective than self-assessment on performance in speech accuracy. Keywords: Peer assessment, self-assessment, speech;
Download free PDF
View PDF
chevron_right
A study on the effectiveness of self-assessment and peer assessment in the oral production of secondary EFL learners and CLIL learners
Yasmina León Herrera
2014
2 Introduction 2 Literature review 3 Research questions and hypotheses 6 Methodology 6 Participants 6 Data collection 7 Results 8 Recording results 8 Placement tests 13 Questionnaires 13 Discussion 14 Conclusions and Further Research 15 Acknowledgements 17 References 18 Appendices 20 Appendix I 20 Appendix II 23 Appendix III 24 Appendix IV 26 Appendix V 27 Appendix VI 28
Download free PDF
View PDF
chevron_right
Self- and Peer-Assessment of Speaking
soo hyoung joo
2016
The research on the second language (L2) speaking assessment has been predominantly concerned with formal proficiency tests. However, with a growing interest in learning-oriented assessment (LOA), more researchers are interested in learner involvement in speaking assessment (Blanche & Merino, 1989; Luoma, 2004; Ocarson, 1989). Self- and peer-assessment of speaking, where learners evaluate the performance of their own and their peers, can be an exemplar of assessment with its primary purpose on learning (Chen, 2006, 2008; Ibersson, 2012; Saito, 2008; Topping & Ehly, 1998). Depending on the purpose, self- and peer-assessment can take various forms: a questionnaire on general speaking ability, a learning log for metacognitive reflection, or a classroom activity where learners use the same rating criteria as their teachers (Bachman & Palmer, 1989; Chen, 2008, Rivers, 2001).
Download free PDF
View PDF
chevron_right
Teachers' Assessment Approaches Regarding EFL Students' Speaking Skill
PROFILE Journal
Profile: Issues in Teachers' Professional Development, 2021
This paper reports an exploratory and descriptive qualitative study on speaking assessment approaches in a teacher education program at a Colombian university. The study aimed to explore how four in-service English language teachers approach the assessment of students' speaking skill. The data were gathered through classroom observations, interviews, and documentary analysis. Results revealed teachers' preference for summative assessment practices to determine students' progress regarding speaking. As a conclusion, teacher professional development in terms of language assessment may be seen as an alternative to develop significant assessment processes where students, teachers, and the institution can be benefited.
Download free PDF
View PDF
chevron_right
590847
research-article2015
LTJ0010.1177/0265532215590847Language TestingBabaii et al.
/$1*8$*(
Article 7(67,1*
Language Testing
Speaking self-assessment:
1–27
© The Author(s) 2015
Reprints and permissions:
Mismatches between learners’ sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/0265532215590847
and teachers’ criteria ltj.sagepub.com
Esmat Babaii, Shahin Taghaddomi and
Roya Pashmforoosh
Kharazmi University, Iran
Abstract
Perceptual (mis)matches between teachers and learners are said to affect learning success or
failure. Self-assessment, as a formative assessment tool, may, inter alia, be considered a means
to minimize such mismatches. Therefore, the present study investigated the extent to which
learners’ assessment of their own speaking performance, before and after their being provided
with a list of agreed-upon scoring criteria followed by a practice session, matches that of
their teachers. In so doing, 29 EFL learners and six EFL teachers served as participants; the
learners were asked to assess their audio-recorded speaking performance before and after
their being provided with the scoring criteria and practice session. The teachers were also
asked to assess the learners’ performance according to the same criteria. Finally, the learners
were required to evaluate the effectiveness of doing self-assessment in the form of reflection
papers. The results revealed a significant difference between the learners’ assessment of their
own speaking ability on the two occasions. The findings also suggested that providing the
learners with the scoring criteria and the follow-up practice session minimized the existing
mismatches between learner assessment and teacher assessment. Moreover, the inductive
analysis of the reflection papers yielded a number of themes suggesting that, despite some
limitations, the learners’ overall evaluation of the effectiveness of speaking self-assessment
was positive.
Keywords
Assessing speaking, evaluation criteria, mismatch, self-assessment, teacher assessment
Coherence is said to be a prerequisite for the success of any curriculum. According to
Johnson (1989), a “coherent curriculum” is one in which all stakeholders, namely policy
makers, syllabus designers, materials developers, teacher trainers, teachers, and learners,
Corresponding author:
Esmat Babaii, Department of Foreign Languages, No. 43, Mofatteh Av., Enghelab Sq., Kharazmi University,
Tehran, 15719-1491, Iran.
Email:
[email protected]
or
[email protected]
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
2 Language Testing
develop a similar understanding of the goals and objectives for which the curriculum is
run in the first place. In this regard, any instance of mismatch, hence, is believed to lead
to, in one way or another, incoherence and, accordingly, failure of the curriculum. More
specifically, Kumaravadivelu (1994), calls attention to possible perceptual mismatches
among teachers and learners, or what he refers to as “teacher intention” and “learner
interpretation”, and their effect on the success and failure of the learning and teaching
processes.
Self-assessment, as a formative assessment tool and an alternative to traditional
modes of assessment, may, inter alia, be considered a means to minimize such mis-
matches. The self-assessment procedure has introduced changes to the students’ learn-
ing experience. Relevant studies recommend using both self- and teacher assessments
in the classroom (Boud, 2013; Falchikov, 2013). It is pointed out that getting learners
involved in assessing their own ability would help them improve learning. Following
the literature on alternatives in assessment, it was found that previous studies com-
pared self- and teacher assessments and examined the reliability of these multiple rat-
ings. Regarding the importance of assessment criteria and evaluation of learners’ and
teachers’ perceptions of common criteria for rating (Orsmond, Merry, & Reiling, 1997,
2000), it seems that the mismatches between learners’ and teachers’ assessments con-
cerning the assessment criteria for rating L2 speaking have remained rather
unaddressed.
Given the prominent status of speaking self-assessment, few studies have been
done to explore the learners’ rating criteria for L2 speaking and their rating accuracy
as measured against those of their teachers. Gaining a better understanding of the
mismatches between multiple ratings (i.e., self- and teacher assessments) requires
empirical research to examine the differences in the learners’ and teachers’ percep-
tions and use of speaking scoring criteria and to discover the impact of employing
assessment criteria on the accuracy of learners’ ratings. The question that still remains
unanswered in the literature on speaking self-assessment is what criteria learners use
to assess their own speaking performance and what can be done to increase the accu-
racy of learners’ self-awarded scores. Since speaking self-assessment involves learn-
ers in assessing their own ability and speaking has received scant attention in this
regard, as Luoma and Tarnanen (2003) pointed out, the main focus of the present
study was whether there is a significant difference between learners’ assessments
before and after their being provided with the rating criteria and the follow-up prac-
tice session. In addition, this study sought to investigate the extent to which the
administration of the scoring criteria affected the congruence between learners’ and
teachers’ assessments.
Literature review
Perceptual mismatch
Given the nature of L2 speaking, the perceptual gaps between learners and teachers are
unavoidable. The concept of “learner perception”, according to Barkhuizen (1998),
involves learners in a decision-making process. Speaking assessment, with specific
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 3
reference to teaching and learning in the language classroom, emanates not just from
learners but from teachers as well. Accordingly, the success of any assessment procedure
also depends on the decisions teachers need to make concerning learners’ performance
and the criteria used for scoring. Differences in assessing speaking ability, however, do
exist regarding how learners perceive the speaking construct and what teachers actually
do in speaking assessments. For example, Kuo’s (2011) study on learner–learner interac-
tion within real classroom context revealed learners’ different perceptions of the factors
contributing to classroom interaction, including communicative intelligibility, gram-
matical accuracy, and corrective feedback.
Kumaravadivelu (1991) suggested that perceptual gaps exist as a result of at least
10 potential sources of mismatches between “teacher intention” and “learner inter-
pretation”, namely, cognitive (knowledge of the world), communicative (message-
oriented communication skills), linguistic (syntactic, semantic, and pragmatic
knowledge of the still developing target language), pedagogic (teachers’ and learn-
ers’ perceptions of stated or unstated objectives of teaching/learning), strategic
(understanding of learning strategies and styles), cultural (prior knowledge of the
target cultural norms), evaluative (articulated and unarticulated modes of self-evalu-
ation measures used by learners to monitor their progress), procedural (paths chosen
by learners to solve problems), instructional (directions given by teachers to help
learners to solve problems), and attitudinal (participants’ disposition towards the
nature of the classroom culture, and teacher and learner role relationships). The gaps
in perception need to be identified and properly handled to promote successful learn-
ing outcomes in the classroom. Previous research on speaking self-assessment has
suggested that learners as reflective practitioners are encouraged to develop more
responsibility when assessing their own performance. Therefore, there is a need to
shed light on the possible evaluative gaps in learners’ and teachers’ perceptions of
speaking ability.
Speaking assessment
The assessment of speaking, as an extremely difficult skill to test, involves a number of
procedures to capture all the defining characteristics for objective testing. An under-
standing of the nature of speaking not only helps define the construct in question, but
ultimately makes it possible to identify factors involved in speaking assessment (Kim,
2010). According to Butler, Eignor, Jones, McNamara, and Suomi (2000, p. 10), for
example, “such features are likely to include accomplishment of task, sufficiency of
response, comprehensibility, adequacy of grammatical resources, range and precision of
vocabulary, fluency, and cohesion.” Performance on each aspect may vary from indi-
vidual to individual and from task to task.
L2 speaking assessment, nowadays, calls for more formative assessment which is a
key to assessment for learning rather than assessment of learning (Nicol & Macfarlane-
Dick, 2006; Stobart, 2008). Regarding the centrality of assessment practices, a col-
laborative effort within the democratic atmosphere of the classroom in which a shared
responsibility for teaching, learning, and evaluating is overarching has been involved
in self-appraisal activities (Butler & Lee, 2006, 2010; Grez & Berings, 2010; Leger,
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
4 Language Testing
2009; Orsmond, Merry, & Callaghan, 2013). Speaking self-assessment, as a formative
assessment tool, promotes learning, establishes a goal-oriented activity, alleviates the
assessment burden on teachers, and finally continues as a long-lasting experience
(Kirby & Downs, 2007; Mok, Lung, Cheng, Cheung, & Ng, 2006; Ross, 2006).
Speaking self-assessment also relies on the social dimension of learning, as Orsmond
and Merry (2013) noted, in which self-directed progressive learning is a key aspect in
the appropriation of the learning objectives and the involvement of teachers and learn-
ers in collaborative learning. To shed light on the notion of learning in self-assessment,
Orsmond (2011) proposed a learner-centered GOALS process. In the GOALS process,
learners’ concerns are to grasp the learning objectives, orient themselves towards self-
regulation, take actions to achieve the desired goals, evaluate their learning, and
develop the necessary strategies to make satisfactory progress. Likewise, an ipsative
approach to self-assessment, as Hughes (2011) suggested, informs learner of the per-
ceived progress in relation to the learner’s previous performance. Under the influence
of the principles of self-regulation and learner autonomy, Nicol and Macfarlane-Dick
(2006) argued that teacher and learner dialogue around the assessment process and the
criteria they apply to evaluate performance can be very helpful. As a result, the defin-
ing characteristic of effective self-assessment is the involvement of learners in under-
standing the criteria for rating speaking and empowering them to close the gaps in their
performance. Further discussion regarding assessment criteria may help avoid any
possible (mis)matches between teachers’ and learners’ perceptions of the criteria.
In line with previous studies on speaking self-assessment (Orsmond, Merry, &
Reiling, 1997, 2000; etc.), it follows that learners’ criteria when assessing their own
speaking ability as examined against teachers’ rating criteria for L2 speaking have
been rather underexplored. In fact, gaining a better understanding of the speaking self-
assessment requires empirical research to discover the impact of speaking scoring cri-
teria and the follow-up practice session on the accuracy of learners’ self-assessment
when compared to teacher assessment. The aim of the present study, therefore, was to
investigate the (mis)matches between learners’ assessment of their own speaking abil-
ity and teachers’ assessment and examine the effectiveness of speaking self-assessment
before and after the learners’ being provided with speaking scoring criteria and the
follow-up practice session. To this end, the following questions constituted the focus
of the study:
1. What criteria do learners use to rate their own speaking performance before their
being provided with the speaking scoring criteria and the follow-up practice
session?
2. Is there any significant difference between the learners’ assessment of their own
speaking ability before and after being provided with the speaking scoring crite-
ria and the follow-up practice session?
3. To what extent are learners’ assessments in agreement with teachers’ assessments
prior to and following the learners’ being provided with the speaking scoring
criteria and the follow-up practice session?
4. How do the learners evaluate the effectiveness of their speaking
self-assessment?
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 5
Table 1. Teachers’ profile summary.
Pseudonym Gender Educational Levels taught Context of Years of
background teaching teaching
experience
T1 Female MA in English language All University/ 8 years
and literature language centers
T2 Female MA in TEFL All University/ 6 years
language centers
T3 Female BA in TEFL Beginner to Language centers 4 years
intermediate
T4 Female BA in TEFL Beginner to upper- Language centers 6 years
intermediate
T5 Female PhD student in TEFL Upper-intermediate University/ 3 years
to advanced language centers
T6 Male PhD student in TEFL All University/ 9 years
language centers
Method
Participants
Twenty-nine EFL learners and six EFL teachers took part in this study. Ranging in age
from 18 to 23 years, the learners were 22 females and seven males. They were first-year
undergraduate students majoring in English language and literature at Kharazmi
University, Tehran, Iran and were enrolled in a conversation course regularly offered to
students of English language and literature at this level. It should be mentioned that none
of the researchers were responsible for delivering the course. Furthermore, in the Iranian
context, nationwide examinations are run yearly to allow entry to university, and, there-
fore, it can be assumed that the participants of the study enjoyed, more or less, the same
level of language proficiency, corresponding roughly to the intermediate level. It should
also be noted that none of the learners had the experience of taking part in self-assess-
ment practice sessions prior to the study.
The teachers, on the other hand, were five females and one male ranging in age from
24 to 33 years with three to nine years of experience in teaching English as a foreign
language. They were all experienced IELTS instructors and had no previous familiarity
with the learner group. Brief profiles of the six teachers are given in Table 1; they are
given pseudonyms to protect their anonymity.
Instruments and data collection procedures
First of all, the participant-teachers were asked to team up and decide on three topics to
elicit the learners’ description, narration, and argumentation speaking abilities. The
learners, then, gathered up in a language laboratory equipped with computerized record-
ing systems, were provided with the three topics about each of which they were required
to talk for three minutes. Following a format similar to the second task of the speaking
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
6 Language Testing
section of the IELTS examination, each topic was followed by a couple of prompts speci-
fying what the learners were supposed to cover in their talk. Provided with the chance to
take notes, the learners were also given one minute to get prepared to talk about each
topic (see Appendix 1).
Having completed all the three tasks, each learner was provided with an audio-
recorded copy of his or her own performance. The learners were, then, given a speaking
assessment sheet requiring them to write down their own personal criteria for rating
speaking ability. They were also asked to listen to their own voice two times and give
themselves a score out of 50 with regard to their performance on each task accordingly
(see Appendix 2).
Subsequently, the researchers asked the participant-teachers, all of whom having the
experience of preparing learners for the speaking section of the IELTS examination, to
develop an agreed-upon list of criteria against which the performance of the learners
could be evaluated. In so doing, it should be emphasized that they had no access to copies
of the learners’ speaking performance. Using their experience as IELTS instructors and
consulting the relevant sources on assessing speaking (e.g., Luoma, 2004), the partici-
pant-teachers attempted to develop their own lists of criteria individually in the first
place. They were then invited to gather up, share their proposals, and come up with a
final list of criteria. What follows, therefore, are the 10 criteria agreed upon by all the six
teachers.
1. Fluency (without pauses, hesitation, and false starts)
2. Grammar (accuracy and variety of structures)
3. Vocabulary (appropriateness and variety of expressions)
4. Pronunciation (stress, rhythm, and intonation)
5. Communicative effectiveness (clarity of ideas and comprehensible (i.e., under-
standable) speech)
6. Topic management (topic relevance, topic coverage, and adequacy of details and
examples)
7. Confidence (anxiety-free speech)
8. Organization (initiation, development, termination and interconnectedness of
ideas)
9. Strategy use (avoiding unfamiliar language and compensating by using familiar
language)
10. Time management (timing your talk)
Building upon the agreed-upon criteria, the researchers developed a checklist com-
prised of 10 items on a five-point Likert scale, yielding a total of 50 (see Appendix 3).
An optimal interval between the two rating occasions seemed to be one long enough
not to let memory effect cloud the participants’ judgments. Moreover, since the partici-
pants were not supposed to have any other systematic exposure to self-assessment
practice sessions, there was a slim chance for other learning experiences to contami-
nate the results. Therefore, with an interval of 40 days, the learners were invited to
collaborate again in the language laboratory. Having provided the learners with the
checklist, the researchers introduced and explained the criteria against which
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 7
the learners were supposed to evaluate their own speaking ability. Secondly, in order to
provide appropriate modeling and enhance the learners’ understanding, three samples of
speaking performance were each played twice and evaluated according to the introduced
checklist; throughout the first two evaluations, which lasted for about 40 minutes, the
researchers, adopting the think-aloud technique, evaluated the samples. Meanwhile, the
participants remained mostly silent although they were given the chance to provide com-
ments and pose questions whenever they desired. For the third evaluation, which lasted
for about 30 minutes, however, the participants were asked to adopt a more active role
and cooperate with the researchers. Finally, the learners were asked to listen to the copies
of their own performance two times, evaluate their own speaking ability according to the
ten criteria included in the checklist, and give themselves a score out of 50.
Meanwhile, the six participant-teachers were asked to meet up in the same language
laboratory; they were provided with copies of the learners’ speaking performance and
asked to rate their speaking ability according to the same checklist, without having access
to the learners’ scores.
Finally, about one month later, the learners were asked to write reflection papers in
English and elaborate on their attitudes towards speaking-assessment in general and
what they experienced in the course of the present study in particular. As mentioned
before, all the learners were enrolled in a conversation course of which self-assessment
was not a regular component. They were only briefly introduced to the idea and practice
of self-assessment for the purpose of the present study. As such, it was attempted to
assure the learners not to feel any pressure to respond positively since the attitudes they
were sharing were not supposed to affect their final course grades at all.
Data analysis
To answer the first two questions, the data were gathered from the two administrations of
the speaking assessment sheet, once before and once after the learners’ being provided
with the speaking scoring checklist and the follow-up practice session. As to the first
question, the criteria the learners mentioned in their comments for rating speaking on the
first occasion were analyzed inductively and grouped into categories. Moreover, regard-
ing the second question, the two sets of quantitative data collected on the two occasions
were, also, entered into the SPSS program and were analyzed by the paired-samples
t-test to discover whether or not significant changes occurred as to the learners’ assess-
ment of their own speaking performance owing to the provision of the speaking scoring
checklist.
As to the third question, the Pearson product–moment correlation was run to investi-
gate any significant relationship between the sets of scores assigned to the learners’
speaking ability by the learners themselves and the teachers on the two occasions. The
two r values were then converted into Fisher’s z-scores so as to investigate whether the
correlations for the ratings prior to and following the criteria were statistically
significant.
Finally, to probe the fourth question, the researchers adopted the Constant Comparative
Method, an inductive and data-driven method used in grounded theory (Glaser & Strauss,
1967), to analyze the qualitative data emerging from the learners’ reflection papers.
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
8 Language Testing
Being widely employed in qualitative studies, the Constant Comparative Method was
considered appropriate to be adopted to extract recurring themes and categories from the
data (Creswell, 2012). First of all, the indicators were identified and coded throughout
the data set. Next, the coded data were grouped into units called themes. The themes, in
turn, were grouped together to form categories. Checking the data against the extracted
codes, themes, and categories continued until full “saturation” was achieved (see
Creswell, 2012). It should be noted that the researchers went through the data analysis
procedure individually. They, then, met up to discuss potential instances of disagreement
and reach a final consensus. Accordingly, the categories and themes presented under the
result section are the ones that the researchers agreed on.
Results
Learners’ self-mentioned criteria for rating speaking
The first research question intended to probe the learners’ criteria to rate their own speak-
ing performance before their being provided with the teachers’ agreed-upon criteria and
the follow-up practice session. Through inductive analysis, the learners’ comments were
compiled and categorized to arrive at the learners’ assessment criteria for L2 speaking.
The emerging criteria were subsequently put under seven categories.
What follows presents the extracted criteria along with follow-up, relevant excerpts
taken from the learners’ comments on their areas of strength and weakness.
1. Topic management:
Recurring in all comments (f = 41), this criterion referred to the adequacy of the answer (i.e.,
topic coverage) while being focused on the topic. L16, for instance, stated that
I’m not completely focused on the topic. I couldn’t manage to answer all the questions.
2. Confidence:
Similarly present in almost all comments (f = 38), the second criterion covered anxiety-free
speech and confidence in speech delivery. In this regard, L8, for example, noted that
I was not prepared for a sudden exam. I was so stressful and I lost my concentration
3. Fluency:
The next major criterion (f = 37) pointed to smooth speech without hesitation and pauses. L6,
for instance, wrote that
There was a gap between my words and I used a sound like Hmm a lot. And the speed of my
speaking was not good.
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 9
4. Time management:
Present in most comments (f = 33), this category encompassed the theme of “keeping the time”.
Echoing the same criterion, L26, for example, commented that
I was supposed to talk 3 minutes. But I couldn’t speak for 3 minutes for each topic.
5. Grammar:
Grammatically correct utterances with the corresponding features of appropriateness and
accuracy was another frequently mentioned criterion (f = 30) by the EFL learners when
assessing their speaking ability in the current research. In this regard, one of the learners (L1),
for example, noted that
I made some grammar mistakes and I couldn’t control the complex grammatical structures in
my speech.
6. Vocabulary:
Several learners (f = 28) also proposed range and variety of lexical choices. The learners were
primarily concerned with the effective use of a wide range of vocabulary in their speaking
performance. L12, for example, stated that
The words that I used were very simple. I also repeated some words. Although I know a lot of
expressions and collocations, I did not use them.
7. Pronunciation:
The final prevailing criterion (f = 25) referred to prosodic features of speaking, including accent
and voice quality. Referring to this criterion for rating speaking, one of the learners (L8)
commented that
The most serious problem was my accent. I spoke in a Farsi accent and this was completely
obvious in my speech.
Learners’ assessment of their own speaking ability
The second research question explored any significant difference between the learners’
assessment of their own speaking ability before and after their being their provided with
the speaking scoring criteria and the follow-up practice session. To address this question,
a paired-samples t-test was employed to investigate the effect of the speaking scoring
criteria on the learners’ ratings. As displayed in Table 2, the learners’ mean scores before
and after their being provided with the criteria are 36.08 and 33.82, respectively. A
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
10 Language Testing
Table 2. Paired-samples statistics for the learners’ ratings.
M N SD SEM
Before the criteria 36.08 29 7.95 1.47
After the criteria 33.82 29 8.31 1.54
Table 3. Paired-samples t-test between learners’ assessment of their speaking ability before
and after being provided with the speaking criteria and the follow-up practice session.
M SD SEM 95% CI t df Sig. (2-tailed)
2.25 3.94 .73 [.75, 3.76] 3.08 28 .005
Table 4. Descriptive statistics of total assessments.
Min Max M SD
Teacher assessment (T) 12.00 46.00 33.27 6.55
Learner assessment (L1) Phase 1 10.00 45.00 36.08 7.95
Learner assessment (L2) Phase 2 10.00 44.00 33.82 8.31
comparison across the two ranges of mean scores reveals that the learners gave them-
selves higher scores on the first occasion than the second.
Table 3 presents the results of the paired-samples t-test applied to the mean scores of the
learners on the two rating occasions. The results show a statistically significant difference
in the learners’ ratings with the t-observed value of 3.08 (p < .01). The mean difference in
the speaking ratings was 2.25 with a 95% confidence interval ranging from .75 to 3.76.
Given the eta-squared value of .25, it can be concluded that there was a large effect size,
with a substantial difference in the learners’ ratings when they were given the criteria.
Learners’ and teachers’ assessments of learners’ speaking ability
The third research question investigated the extent of agreement between the learners’
and teachers’ assessment prior to and following the learners’ being provided with the
criteria and the follow-up practice session. As shown in Table 4, to analyze differences
of ratings between the teachers and learners, the mean scores, maximum and minimum
scores, and standard deviations were calculated between the two groups. The mean
scores for the teachers and learners do not show a large difference following the scoring
criteria. The difference between the two mean scores in the second phase is just .55
(L2−T), indicating an increase in agreement between the learners’ and teachers’ ratings.
The next step in the analysis was to calculate the correlations between teacher assess-
ment and self-assessment on the two occasions. The scores reported in the current study
for the teachers were the average scores given by all the six teachers with the inter-rater
reliability of .81. The correlation between the scores awarded by the learners and teach-
ers was estimated using the Pearson product–moment correlation coefficient. Preliminary
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 11
Table 5. Pearson product–moment correlations between learners’ and teachers’ ratings.
Teacher Learner Learner
assessment assessment assessment
Phase 1 Phase 2
Teacher Pearson correlation 1 .730** .901**
assessment Sig. (2-tailed) .000 .000
N 29 29 29
**Correlation is significant at the .01 level (2-tailed).
analyses were performed to ensure no violation of assumptions, including normality,
linearity, and homoscedasticity. Table 5 gives the correlation indices. The resulting coef-
ficient for the learners, before their being provided with the criteria, and the teachers was
found to be .73 which is significant at p < .01 level.
Furthermore, as displayed in Table 5, the correlation coefficient between the teachers’
ratings and the learners’ self-awarded scores following the scoring criteria points to a
strong, positive correlation (r = .90), which is also significant at the p < .01 level.
The correlation coefficients between the teachers’ and the learners’ evaluation of the
learners’ speaking ability on the two occasions are statistically significant. Moreover,
building upon the effect size criteria developed by Cohen (1988), proposing that a
Pearson correlation coefficient larger than .50 is considered to be a large one, it can be
argued that the correlation between the teachers’ and the learners’ evaluation was high
even when the learners had no access to the criteria. However, when squared, the first
correlation coefficient (r1 = .73) yields .53, indicating 53% shared variance. The coeffi-
cient of determination for the second coefficient (r2 = .90), on the other hand, is .81 and
shows 81% of shared variance which is noticeably larger than 53%. Furthermore,
Fisher’s z-transformation was employed to convert the two r values into z-scores and,
accordingly, compare the statistical difference between the two correlation coefficients.
The result of the comparison (zr1 = .93, zr2 = 1.53, n = 29) was found to be −2.17 (CI =
95%), indicating that the coefficients were significantly different.
This implies a significant increase in agreement between the learners’ and teachers’
scores following the provision of the scoring criteria and the follow-up practice session.
The effectiveness of the speaking self-assessment
The fourth question aimed at probing the learners’ attitudes regarding the effectiveness
of their speaking self-assessment. The answer to this question emerged from the analysis
of the reflection papers written by the learners. The major themes deriving from the data
were as follows: (1) increased self-awareness and detection of weak points followed by
improved learning; (2) the positive influence of the use of speaking scoring criteria on
the accuracy of self-assessment; (3) the long-lasting effect of self-assessment, in com-
parison to teacher assessment, on learning; (4) the unreliable nature of self-assessment;
(5) the time-consuming nature of self-assessment; (6) encouraging ongoing conduction
of self-assessment; and (7) providing perfect models.
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
12 Language Testing
In the following paragraphs, these themes, presented under three main categories
(i.e., the benefits of speaking self-assessment, the limitations of speaking self-assess-
ment, and suggestions for the betterment of self-assessment) are elaborated on. It should
be pointed out in advance that no changes were made to the excerpts taken from the
learners’ reflection papers with regard to grammar, punctuation, and so on, except
where a change seemed absolutely necessary to rule out any possibility of misunder-
standing. It should be also noted that in order to guarantee the learners’ anonymity, they
were given pseudonyms.
Benefits of speaking self-assessment
Increased self-awareness and detection of weak points followed by improved learning. In one
way or another, all of the learners (f = 29) commented that the speaking self-assessment
was helpful since it drew their attention to their weaknesses in speaking English and offered
them the chance to work on and minimize those weaknesses. In this regard, one of the learn-
ers (L18), for example, stated that
I think it helps us to evaluate our own speaking ability and be able to fix our problems. I found
out what errors do I make the most while speaking and what parts I should focus on to speak
better.
Similarly, L15 and L24, respectively, commented that
Listening to your speaking lets you know your errors … It is helpful because you can understand
your mistakes and you can try to avoid them by practicing.
I think it is very useful and we can improve our speaking skill by understanding our weaknesses.
I could understand my speaking weaknesses by listening to my recorded voice and it was very
helpful for me to think about them and write them on a paper.
Moreover, referring to their own current, actual experience with self-assessment, L20,
highlighting the importance of the checklist they were provided with, expressed his ideas
in the following words.
As soon as we heard the recordings and listened to our own voice, we understood we have a lot
of problems in our speaking. So we started doing our best in order to eliminate and solve these
speaking problems of ours.
Some of them went even further and claimed that weakness detection and, accordingly,
improvement in speaking proficiency may not be possible unless one listens to his or her
own voice. L9, for instance, pointing to her own detection of her areas of difficulty
through self-assessment, wrote that
Until we can’t hear our own voice it’s very obvious that we can’t get our mistakes and mistakes
can make progress. Now I got that I don’t have a good pronunciation and I will surely work on it.
Echoing the same ideas, L8 and L7, respectively, stated that
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 13
It helped me to know my weaknesses. For example, when I listened to my audio-recorded
speech, I found out that I have to improve my accent.
It let me hear my voice to decide how to speak better. Sometimes I need to raise the intonation
or fall it. Or I just figured out that there is vibration in my voice which actually needs to be
overcome.
The positive influence of the use of speaking scoring criteria on the accuracy of self-assessment.
As to the benefits of speaking self-assessment, the second major theme (f = 20) that
emerged from the data was concerned with the learners’ positive evaluation of their being
provided with the speaking scoring criteria in the second phase. L26, for example, com-
paring the first and the second phase, commented that
[In the case of the first phase] I didn’t know if my scoring was fair enough because my speaking
criteria may be different from my teacher’s. Besides what matters the most for me, may matters
the least for [the teacher]. [However, the second phase] gave me a clearer outlook of the
required elements of how to give a good speech. It was very helpful to learn about new criteria
of speaking fluently because I can work on new, crucial factors that I now know of.
Explaining how the speaking scoring criteria made her aware of all the factors leading to
standard, effective speech, the same learner went on and added
I realized that giving a good speech is not only speaking with [standard] accent! You need a lot
of other factors. You have to be able to organize your sentences; you need a good vocabulary
and good information about the topic of discussion.
Similarly, L11, emphasizing that the speaking scoring criteria encouraged them to have
a more realistic assessment of their own speaking proficiency, stated that
Analyses by listeners were more detailed, so we would have a more careful assessment and
better scoring system. Here the required factors were mentioned so the assessments were nearer
to real results.
Along the same line, with respect to the practicality of speaking self-assessment when
they had access to the speaking scoring criteria, L16 and L12, respectively, wrote that
In comparison with phase one, the scoring system was more efficient I guess and the criteria
which were more clearly mentioned. They were more detailed so the scoring was easier.
I think, realistically, I can pay more attention to and focus on the criteria which I wasn’t
concerned before. I think it is more practical than phase one because there may be some other
criteria that should be regarded, but we didn’t know anything about them in phase one.
The long-lasting effect of self-assessment, in comparison to teacher assessment, on learn-
ing. Present in most of the reflection papers (f = 16), the third major theme under this
category referred to the positive long-term effect of doing self-assessment on learn-
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
14 Language Testing
ing. Taking into account the context of the present study where final course grades are
reported in the absence of any formative feedback, the learners believed that they would
not be provided with the chance of being aware of and working on their typical areas of
difficulty. In their views, teacher assessment, in contrast with self-assessment, may not
have played a positive, active role in the process of their learning. L11, for example,
stated that
When you review your task, you can find your error. While finding your error, you can resist
making it again, but if you take your exam result, you won’t be able to find out what is your
fault … When the exam is gone, we care about the result, not our mistakes.
Some of the learners went even further and claimed that even if they were provided with
feedback by their teachers, they would not be very likely to take maximum advantage of
such feedbacks in the long-run. L12, for instance, stated that
Definitely, it’ll inform me of something that I could have done but by the self-assessment, this
will stick in my mind more efficient than when it’s remarked by someone else.
Furthermore, L4 commented that
You know. I don’t remember how many times I received feedback to my speaking from my
professors. Even if I did, I don’t remember. But this experience was wonderful. I never cared
about the organization of my speaking. You know. But now I am going to pay attention to it
whenever I am going to talk.
Limitations of speaking self-assessment. Although all of the learners had something posi-
tive to say about speaking self-assessment in general and their experience with doing
self-assessment of their own speaking proficiency in particular, some of them pointed
out some limitations of self-assessment that, to them, may have compromised its bene-
fits. Two major themes under this category are presented below.
The unreliable nature of self-assessment. Some learners (f = 11) believed that as a result
of learners’ lack of expertise in language assessment, self-assessment is unreliable in
nature. They felt the desperate need for teachers’ assessment of their speaking perfor-
mance. L16, for example, commented that
[The disadvantage of self-assessment is the lack of] being assessed and commented by a
specialist who can see all aspects better and can criticize based on standards.
Likewise, casting doubt on their capability to self-assess, even after being aware of the
relevant criteria, L25 and L15, respectively, commented that
Because we are not aware of the exact criteria of scoring, it would be more helpful, if someone
else gave us a score.
I think if there was a teacher assessment too, it could be better. I didn’t know if my scoring was
fair enough because what matters the most for me may matter the least for my teacher.
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 15
In the same vein, some learners pointed out that there is a general tendency among language
learners to overestimate their own abilities when they do self-assessment, a problem adding
to the unreliability of self-assessment. Regarding this, L13 and L20, respectively, wrote that
Sometimes there are students who have a high self-confidence and they give themselves a high
score without paying attention their problems.
Perhaps one of the most common disadvantages in such procedures is that some students score
themselves more than what they deserve. [So,] instructors should not rely only on students’ self-
assessment for final scoring.
The time-consuming nature of self-assessment. Under this category, the second theme
emerging from the data (f = 5) referred to the time-consuming nature of self-assessment.
For example, L11 and L18, respectively, commented that
In my idea, it was time consuming for students, but generally it didn’t have much disadvantages.
There were no disadvantages although it was a little time consuming.
Suggestions for the betterment of self-assessment. The last category was concerned with
the suggestions put forward by the learners for the betterment of self-assessment. In this
regard, two major themes extracted from the data were as follows.
Encouraging ongoing conduction of self-assessment. The first theme (f = 14) showed that
the learners believed that, for self-assessment to be beneficial, it has to be an ongoing
process and done on a regular basis. L13 and L9, respectively, were learners whose
reflection papers provided evidence for this theme.
It should be repeated regularly with the better standard and it can help the students to progress
more and more.
I think just one time is very low for this practice. It should be continued so we can see if we had
improved or not.
Providing perfect models. The other theme under this category (f = 6) revealed the
learners’ desire to be provided with perfect models of language performance under the
assumption that exposure to such models helps them get familiar with all aspects of
effective speech and paves the way for them to become like those models in the long-run.
L20, for instance, wrote that
… to present a practical standard example. Maybe a recording of an expert that knows the
criteria and performs according to them can be helpful.
Discussion
The results indicated that a number of linguistic and non-linguistic criteria encompassed
both the learners’ and the teachers’ mentioned criteria for rating speaking. The analysis
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
16 Language Testing
of the comments the learners wrote when assessing their own ability before their being
provided with the criteria showed the learners were more concerned with topic manage-
ment, confidence, fluency, time management, grammar, vocabulary, and pronunciation.
However, they failed to point to macro-level components, like organization, strategy use,
and communicative effectiveness, included in the list of the teachers’ agreed-upon crite-
ria with which they were provided on the second occasion. Based on the findings of the
present study, it appeared that the teachers’ criteria were compatible with those reported
in previous studies (e.g., Iwashita, Brown, McNamara, & O’Hagan, 2008; Plough,
Briggs, & Van Bonn, 2010; Zhang & Elder, 2011). The learners’ self-mentioned criteria,
on the other hand, suggested that the skills-and-components-based perspective made
them lose sight of higher-order speaking assessment criteria in their self-awarded ratings.
This, therefore, in line with previous research (e.g., Orsmond, Merry, & Reiling, 1997,
2000), reveals that the learners were not able to make sound judgments about their own
ability prior to the application of the assessment criteria.
Furthermore, the analysis of the learners’ assessments showed significant differences
between their self-ratings prior to and following their being provided with the assess-
ment criteria and the follow-up practice session. This finding lends support to the claim
that the application of different rating criteria may lead to lack of precision in speaking
assessment (Chalhoub-Deville & Wigglesworth, 2005). In addition, it can be argued that
providing the learners’ with the teachers’ agreed-upon criteria, along with the follow-up
practice session, seems to have led to a better understanding of the importance of the
factors required for more consistent rating, a more comprehensive view of speaking abil-
ity, and, in turn, a narrowed gap between “teacher intention” and “learner interpretation”
(Kumaravadivelu, 1991).
Meanwhile, it was found that the learners rated their ability relatively higher on the
first occasion than they did on the second occasion. The overestimation of the self-ability
could be ascribed to the application of the learners’ personal criteria on the first occasion.
In line with the existing literature (e.g., Jafarpur, 1991; Saito, 2000), it is, then, proposed
that the correlations of learners’ and teachers’ ratings in speaking assessment are rela-
tively low when learners apply their own personal criteria for self-rating.
With regard to the third question, it was found that the correlation between learners’
and teachers’ assessments significantly increased after the learners were provided with
the scoring criteria and the follow-up practice session. It can, then, be suggested, in line
with previous literature (Chen, 2008; Rust, Price, & O’Donovan, 2003, etc.), that the
learners’ enhanced understanding of speaking scoring criteria may bring about improved
learning and higher self-assessment accuracy.
Recent explorations in learners’ perceptions suggest that there exists a gap between
what teachers actually do and how what they do is perceived by their learners (Higgins,
Hartley, & Skelton, 2002; Orsmond & Merry, 2011; Wong, 2009). The present study
revealed that the provision of speaking scoring criteria contributes to the convergence
between learners’ and teachers’ assessments. The findings of this study indicated that
providing learners with the assessment criteria may be a step forward with regard to
minimizing the potential evaluative mismatches, in response to the call made by
Kumaravadivelu (1994), between teachers and learners. The narrower the gaps between
teacher intention and learner interpretation, the greater are the chances of achieving
speaking rating accuracy.
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 17
The answer to the fourth question came from the analysis of the qualitative data deriv-
ing from the learners’ reflection papers. Seven major themes were identified under three
main categories, namely benefits of self-assessment, limitations of self-assessment, and
suggestions for the betterment of self-assessment. With regard to the benefits of self-
assessment, literally all of the learners evaluated their experience with assessment of
their own speaking proficiency positively and commented that it gave them the opportu-
nity to gain a better understanding of their areas of difficulty and, accordingly, embark
on making improvements. This theme, consistent with the existing related literature
(Leger, 2009; Orsmond & Merry, 2011; Patri, 2002; to name but a few) provides further
support for the claim that self-assessment, as an alternative to traditional modes of
assessment, is an efficacious method of raising learners’ self-awareness and facilitating
their learning. This theme also resonates with the basic premises of the GOALS process
(Orsmond, 2011), highlighting the point that the developmental aspects of learners’ self-
directed learning rely on social rather than cognitive dimensions. According to this
model, learners’ self-assessment, as a social practice, leads to the betterment of the learn-
ing experience.
Continuing with the benefit of self-assessment, the second theme referred to the role
the speaking scoring criteria played in raising the learners’ understanding of the building
blocks of standard, effective speech and leading them towards having more realistic
assessments of their abilities. This theme, in line with previous research (e.g., Cheng &
Warren, 2005; Shimura, 2006), indicates that learners’ having access to the relevant cri-
teria results in increased accuracy of self-assessment. Moreover, this theme yields fur-
ther evidence that providing learners with the standard criteria may contribute to
minimizing the possible evaluative mismatches (to use Kumaravadivelu’s words, 1994)
that exist between learners and teachers.
The final theme emphasizing the benefits of self-assessment referred to the learners’
positive evaluation of the long-lasting effect of doing self-assessment on their learning.
Echoing the same ideas, several studies in the literature (e.g., Boud & Falchikov, 2006;
Cheng & Warren, 2005; Orsmond, Merry, & Reiling, 2000) have come to the conclusion
that self-assessment, as a tool for formative assessment, is far more effective than teacher
assessment when it comes to long-term learning. Boud (2000), for example, suggests that
sustainable self-assessment as a self-learning activity, results in improved short- and
long-term learning.
Despite their overall positive evaluation of self-assessment, a few learners, how-
ever, highlighted some limitations that might compromise its benefits. The first theme
under this category shed light upon the learners’ skepticism as to the reliability of self-
assessment. Feeling the overwhelming need for teachers’ assessment of and passing
judgment on their speaking proficiency, the learners suggested that learners’ assess-
ment of their own performance is, more often than not, unreliable and, in most cases,
overestimating. This theme, in harmony with previous findings in the literature (Leger,
2009; Matsuno, 2009; etc.), suggests that self-assessment, as a formative assessment
tool, is not likely to be an appropriate candidate to be employed for summative
purposes.
Moreover, the second theme in this regard reflected the learners’ relative discontent
with the time-taking nature of doing self-assessment. In line with previous research (e.g.,
Hanrahan & Isaacs, 2001), this theme lends support to the hypothesis that learners,
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
18 Language Testing
despite their overall positive evaluation of the benefits of self-assessment, may be dis-
heartened to do self-assessment more frequently owing to its time-consuming nature.
However, the GOALS process (Orsmond, 2011) emphasizes that the time-consuming
nature of self-assessment, as a situated learning practice, is unavoidable considering the
fact that learning is integrated into social practice in which externally taught learning is
a prerequisite for internal learning.
Yet, under the third category, which was concerned with the suggestions made by the
learners to maximize the benefits of doing self-assessment, it was proposed that, for
learners to enjoy the full benefits of self-assessment, they need to do it on a very regular
basis over a long period of time. This suggestion seems to be supported by previous find-
ings in the literature (Leger, 2009, to name one). According to Hughes (2011), for
instance, the central role of self-assessment in terms of the long-term progress has been
linked to ipsative assessment, which is likely to account for how self-regulated learners
have advanced since the previous assessments. Similarly, Boud (2000) discussed the
importance of self-assessment in a view of “learning society”, suggesting that self-
directed learning needs to be active to encourage progressive learning. In the context of
learning-oriented assessment, Boud and Falchikov (2006) also argued for the need to
align assessment with the goal of fostering long-term learning.
The other suggestion put forward by the learners was that learners should be exposed to
perfect models of speaking performance under the assumption that exposure to such models
deepens their understanding of the factors contributing to standard, effective speech. This
suggestion appears to be in harmony with previous findings in the literature, indicating that
introducing learners to exemplars contributes to their improved learning (see Handley &
Williams, 2011; Hendry, Bromberger, & Armstrong, 2011; Orsmond, Merry, & Reiling,
2002).
Conclusion
Developing a shared understanding of the goals and objectives of a given curriculum is
believed to contribute greatly to the coherence and, as such, success of that curriculum
(Johnson, 1989; Kumaravadivelu, 1994). The findings of the present study revealed that
giving learners the chance to do self-assessment according to an agreed-upon set of scor-
ing criteria is an effective way to increase learners’ and teachers’ agreement on how to
rate learners’ speaking proficiency, minimize perceptual mismatches, or more specifi-
cally “evaluative mismatches”, to use Kumaravadivelu’s (1994) exact words, between
them, and, hence, contribute to the coherence and success of the curriculum.
The findings of this study bear both theoretical and practical implications. At the theo-
retical level, with specific focus on the under-researched area of speaking self-assess-
ment, this study may be considered a pioneering step in providing empirical evidence on
the effectiveness of doing self-assessment in minimizing “evaluative” mismatches
between teachers and learners. At the practice level, an implication for teachers is to
provide learners with the opportunity to take part in an ongoing procedure of self-assess-
ment to monitor and enhance their learning over time (Leger, 2009) and to accept owner-
ship for their successes and failures in learning (Tremblay & Gardner, 1995). The
pedagogic benefits of involving students in self-assessment have been confirmed by both
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 19
teachers and learners in the related literature (Butler & Lee, 2006, 2010; Chen, 2006;
Cheng & Warren, 2005). A further implication for major ELT stakeholders, namely pol-
icy makers, syllabus designers, materials developers, teacher trainers, and teachers, is to
contribute to the success of the teaching and learning process by developing and sharing
comprehensive and comprehensible lists of scoring criteria.
However, this study was not without limitations which may have compromised the
generalizability of the findings. First, the data were mainly gathered from a convenient
sample of 29 Iranian English language learners and six Iranian English language teach-
ers. To lend support to the findings of this study, future research, thus, may be done with
larger and more representative groups of participants in diverse contexts. Moreover, pro-
ficiency level was not taken into account as a variable in this study. As evidenced in the
literature, instructors are, more often than not, reluctant to rely on learners’ self-ratings
owing to the unreliable nature of these ratings. The unreliability of self-assessment might
be a result of learners’ lack of language proficiency (Shimura, 2006); therefore, it seems
that the effect of learners’ proficiency level on the accuracy with which they do self-
assessment needs further investigation. Furthermore, in this study, the learners’ assess-
ment of their own speaking proficiency was limited to a particular task type, that is,
monologue. Future research is, then, suggested to explore the same research questions,
employing varied speaking task types. Further research on other areas of language profi-
ciency is also needed to validate the claim that requiring learners to do self-assessment is
an effective way to minimize perceptual mismatches between teachers and learners in
particular and contributes to the success of the curriculum in general.
Funding
This research received no specific grant from any funding agency in the public, commercial, or
not-for-profit sectors.
References
Barkhuizen, G. (1998). Discovering learners’ perceptions of ESL classroom teaching/learning
activities in South African context. TESOL Quarterly, 32(1), 85–108.
Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies
in Continuing Education, 22(2), 151–167.
Boud, D. (2013). Enhancing learning through self-assessment. New York: Routledge.
Boud, D., & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment &
Evaluation in Higher Education, 31(4), 399–413.
Butler, F. A., Eignor, D., Jones, S., McNamara, T., & Suomi, B. K. (2000). TOEFL 2000 speak-
ing framework: A working paper (ETS Research Report No. RM-00–06). Princeton, NJ:
Educational Testing Service.
Butler, Y. G., & Lee, J. (2006). On-task versus off-task self-assessment among Korean elemen-
tary school students studying English. The Modern Language Journal, 90(4), 506–518.
Butler, Y. G., & Lee, J. (2010). The effects of self-assessment among young learners of English.
Language Testing, 27(1), 5–31.
Chalhoub-Deville, M., & Wigglesworth, G. (2005). Rater judgment and English language speak-
ing proficiency. World Englishes, 24(3), 383–391.
Chen, Y. M. (2006). Peer- and self-assessment for English oral performance: A study of reliability
and learning benefits. English Teaching and Learning, 30(4), 1–22.
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
20 Language Testing
Chen, Y. M. (2008). Learning to self-assess oral performance in English: A longitudinal case
study. Language Teaching Research, 12(2), 235–262.
Cheng, W., & Warren, M. (2005). Peer assessment of language proficiency. Language Testing,
22(1), 93–121.
Cohen, J. W. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale,
NJ: Lawrence Erlbaum.
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative
and qualitative research (4th ed.). New York: Pearson.
Falchikov, N. (2013). Improving assessment through student involvement: Practical solutions for
aiding learning in higher and further education. New York: Routledge.
Glaser, B., & Strauss, A. (1967). The discovery of ground theory: Strategies for qualitative
research. Chicago, IL: Adline.
Handley, K., & Williams, L. (2011). From copying to learning: Using exemplars to engage stu-
dents with assessment criteria and feedback. Assessment & Evaluation in Higher Education,
36(1), 95–108.
Hanrahan, S. J., & Isaacs, G. (2001). Assessing self- and peer-assessment: The students’ views.
Higher Education Research and Development, 20(1), 53–70.
Hendry, G. D., Bromberger, N., & Armstrong, S. (2011). Constructive guidance and feedback for
learning: The usefulness of exemplars, marking sheets and different types of feedback in a
first year law subject. Assessment & Evaluation in Higher Education, 36(1), 1–11.
Higgins, R., Hartley, P., & Skelton, A. (2002). The conscientious consumer: Reconsidering
the role of assessment feedback in student learning. Studies in Higher Education, 27(1),
53–64.
Hughes, G. (2011). Towards a personal best: A case for introducing ipsative assessment in higher
education. Studies in Higher Education, 36(3), 353–367.
Iwashita, N., Brown, A., McNamara, T., & O’Hagan, S. (2008). Assessed levels of second lan-
guage speaking proficiency: How distinct? Applied Linguistics, 29(1), 24–49.
Jafarpur, A. (1991). Can naive EFL learners estimate their own proficiency? Evaluation and
Research in Education, 5(3), 145–157.
Johnson, R. K. (1989). The second language curriculum. Cambridge: Cambridge University Press.
Kim, H. G. (2010). Investigating the construct validity of a speaking performance test. Spaan
Fellow Working Papers in Second or Foreign Language Assessment, 8, 1–30.
Kirby, N. F., & Downs, C. T. (2007). Self-assessment and the disadvantaged student: Potential for
encouraging self-regulated learning? Assessment & Evaluation in Higher Education, 32(4),
475–494.
Kumaravadivelu, B. (1991). Language learning tasks: Teacher intention and learner interpretation.
ELT Journal, 45(2), 98–107.
Kumaravadivelu, B. (1994). The postmethod condition: (E)merging strategies for second/foreign
language teaching. TESOL Quarterly, 28(1), 27–48.
Kuo, I. (2011). Student perceptions of student interaction in a British EFL setting. ELT Journal,
65(3), 281–290.
Leger, D. S. (2009). Self-assessment of speaking skills and participation in a foreign language
class. Foreign Language Annals, 42(1), 158–178.
Luoma, S. (2004). Assessing speaking. Cambridge: Cambridge University Press.
Luoma, S., & Tarnanen, M. (2003). Creating a self-rating instrument for second language writing:
From idea to implementation. Language Testing, 20(4), 440–465.
Matsuno, S. (2009). Self-, peer-, and teacher-assessments in Japanese university EFL writing
classrooms. Language Testing, 26(1), 75–100.
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 21
Mok, M. M. C., Lung, C. L., Cheng, D. P. W., Cheung, R. H. P., & Ng, M. L. (2006). Self-
assessment in higher education: Experience in using a metacognitive approach in five case
studies. Assessment & Evaluation in Higher Education, 31(4), 415–433.
Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A
model and seven principles of good feedback practice. Studies in Higher Education, 31(2),
199–218.
Orsmond, P. (2011). Self- and peer-assessment: Guidance on practice in the biosciences. Leeds:
The Higher Education Academy Center for Biosciences.
Orsmond, P., & Merry, S. (2011). Feedback alignment: Effective and ineffective links between
tutors’ and students’ understanding of coursework feedback. Assessment & Evaluation in
Higher Education, 36(2), 125–136.
Orsmond, P., & Merry, S. (2013). The importance of self-assessment in students’ use of tutors’
feedback: A qualitative study of high and non-high achieving biology undergraduates.
Assessment & Evaluation in Higher Education, 38(6), 737–753.
Orsmond, P., Merry, S., & Callaghan, A. (2013). Communities of practice and ways to learn-
ing: Charting the progress of biology undergraduates. Studies in Higher Education, 38(6),
890–906.
Orsmond, P., Merry, S., and Reiling, K. (1997). A study in self-assessment: Tutor and students’
perceptions of performance criteria. Assessment & Evaluation in Higher Education, 22(4),
357–368.
Orsmond, P., Merry, S., & Reiling, K. (2000). The use of student derived marking criteria in peer
and self-assessment. Assessment & Evaluation in Higher Education, 25(1), 23–38.
Orsmond, P., Merry, S., & Reiling, K. (2002). The use of exemplars and formative feedback when
using student derived marking criteria in peer and self-assessment. Assessment & Evaluation
in Higher Education, 27(4), 309–323.
Patri, M. (2002). The influence of peer feedback on self- and peer-assessment of oral skills.
Language Testing, 19(2), 109–131.
Plough, I. C., Briggs, S. L., & Van Bonn, S. (2010). A multi-method analysis of evaluation criteria
used to assess the speaking proficiency of graduate student instructors. Language Testing,
27(2), 235–260.
Ross, J. A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment,
Research, and Evaluation, 11(10), 1–13.
Rust, C., Price, M., & O’Donovan, B. (2003). Improving students’ learning by developing their
understanding of assessment criteria and processes. Assessment & Evaluation in Higher
Education, 28(2), 147–164.
Saito, H. (2000). Peer, self, and instructor ratings of group presentations in EFL classrooms: A
pilot study. The Journal of Rikkyo University Language Center, 2, 76–86.
Shimura, M. (2006). Peer- and instructor assessment of oral presentations in Japanese university
EFL classrooms: A pilot study. Waseda Global Forum, 3, 99–107.
Stobart, G. (2008). Testing times: The uses and abuses of assessment. London: Routledge.
Tremblay, P. F., & Gardner, R. C. (1995). Expanding the motivation construct in language learn-
ing. Modern Language Journal, 79(4), 505–520.
Wong, R. M. H. (2009). Try to describe the main point of your lesson: Student perception and
identification of learning objectives in English lessons. Reflections on English Language
Teaching, 8(2), 73–88.
Zhang, Y., & Elder, C. (2011). Judgments of oral proficiency by non-native and native English
speaking teacher raters: Competing or complementary constructs? Language Testing, 28(1),
31–50.
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
22 Language Testing
Appendix 1
Speaking tasks.
Direction: You are given 3 topics about each of which you are required to talk for 3 minutes.
You have 3 minutes to get prepared. The topics are very general and related to your personal
experience. The examiner will remain silent while you are speaking.
Topic #1:
Describe a house which you would like to buy.
You should say:
•• where it is located
•• how you would describe it in terms of structure (e.g., number of rooms, etc.)
•• and how you would describe it in terms of decoration (e.g., equipment, furniture, etc.)
Topic #2:
Talk about a turning point in your life.
You should say:
•• what exactly happened to you
•• when and where it happened
•• how it affected your life
•• and in what ways you have changed since then
Topic #3:
Describe a serious problem in your hometown/city.
You should say:
•• what it is
•• why you think it is serious
•• what has caused this problem
•• what can be done about it
•• and who may object to this solution/and how you would convince him/her
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 23
Appendix 2
Speaking assessment sheet.
Direction: You were given 3 topics about each of which you were required to talk for 3
minutes. Please, first, write down your personal scoring criteria. Next, listen to each audio-
taped performance, give yourself a score out of 50, and, finally, in the comments section,
comment on your strengths and weaknesses.
Personal Scoring Criteria
……………………………………………………………………………………………………
……………………………………………………………………………………………………
……………………………………………………………………………………………………
……………………………………………………………………………………………………
……………………………………………………………………………………………………
Topic #1:
Comments:
---------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------
Your score: -----------------------/50
Topic #2:
Comments:
---------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------
Your score: -----------------------/50
Topic #3:
Comments:
---------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------
Your score: -----------------------/50
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
24 Language Testing
Appendix 3
Speaking self-assessment sheet.
Direction: You were given 3 topics about each of which you were required to talk for 3
minutes. Then, you listened to each audio-taped performance and gave yourself a score out of
50. Once more, please listen to each audio-taped performance and assess your speaking ability
based on the following speaking criteria. Then, in the last section, assess your overall speaking
ability.
Topic #1:
Describe a house which you would like to buy.
You should say:
•• where it is located
•• how you would describe it in terms of structure (e.g., number of rooms, etc.)
•• and how you would describe it in terms of decoration (e.g., equipment, furniture, etc.)
Speaking Scoring Criteria
1. Fluency (without pauses, hesitation, and false starts)
5 4 3 2 1
2. Grammar (accuracy and variety of structures)
5 4 3 2 1
3. Vocabulary (appropriateness and variety of expressions)
5 4 3 2 1
4. Pronunciation (stress, rhythm, and intonation)
5 4 3 2 1
5. Communicative effectiveness (clarity of ideas and comprehensibility (i.e.,
understandability) of speech)
5 4 3 2 1
6. Topic Management (topic relevance, topic coverage, and adequacy of details and
examples)
5 4 3 2 1
7. Confidence (anxiety-free speech)
5 4 3 2 1
8. Organization (initiation, development, termination / interconnectedness of ideas)
5 4 3 2 1
9. Strategy Use (avoiding unfamiliar language and compensating by using familiar language)
5 4 3 2 1
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 25
10. Time Management (timing your talk)
5 4 3 2 1
Score: -----------------------/50
Topic #2:
Talk about a turning point in your life.
You should say:
•• what exactly happened to you
•• when and where it happened
•• how it affected your life
•• and in what ways you have changed since then
Speaking Scoring Criteria
1. Fluency (without pauses, hesitation, and false starts)
5 4 3 2 1
2. Grammar (accuracy and variety of structures)
5 4 3 2 1
3. Vocabulary (appropriateness and variety of expressions)
5 4 3 2 1
4. Pronunciation (stress, rhythm, and intonation)
5 4 3 2 1
5. Communicative effectiveness (clarity of ideas and comprehensibility
(i.e., understandability) of speech)
5 4 3 2 1
6. Topic Management (topic relevance, topic coverage, and adequacy of details and
examples)
5 4 3 2 1
7. Confidence (anxiety-free speech)
5 4 3 2 1
8. Organization (initiation, development, termination / interconnectedness of ideas)
5 4 3 2 1
9. Strategy Use (avoiding unfamiliar language and compensating by using familiar language)
5 4 3 2 1
10. Time Management (timing your talk)
5 4 3 2 1
Score: -----------------------/50
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
26 Language Testing
Topic #3:
Describe a serious problem in your hometown/city.
You should say:
•• what it is
•• why you think it is serious
•• what has caused this problem
•• what can be done about it
•• and who may object to this solution/and how you would convince him/her
Speaking Scoring Criteria
1. Fluency (without pauses, hesitation, and false starts)
5 4 3 2 1
2. Grammar (accuracy and variety of structures)
5 4 3 2 1
3. Vocabulary (appropriateness and variety of expressions)
5 4 3 2 1
4. Pronunciation (stress, rhythm, and intonation)
5 4 3 2 1
5. Communicative effectiveness (clarity of ideas and comprehensibility (i.e.,
understandability) of speech)
5 4 3 2 1
6. Topic Management (topic relevance, topic coverage, and adequacy of details and
examples)
5 4 3 2 1
7. Confidence (anxiety-free speech)
5 4 3 2 1
8. Organization (initiation, development, termination / interconnectedness of ideas)
5 4 3 2 1
9. Strategy Use (avoiding unfamiliar language and compensating by using familiar language)
5 4 3 2 1
10. Time Management (timing your talk)
5 4 3 2
Score: -----------------------/50
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
Babaii et al. 27
I assess my overall speaking ability to be:
Superior Advanced Intermediate Novice
Speaking Scoring Criteria
1. Fluency (without pauses, hesitation, and false starts)
5 4 3 2 1
2. Grammar (accuracy and variety of structures)
5 4 3 2 1
3. Vocabulary (appropriateness and variety of expressions)
5 4 3 2 1
4. Pronunciation (stress, rhythm, and intonation)
5 4 3 2 1
5. Communicative effectiveness (clarity of ideas and comprehensibility (i.e.,
understandability) of speech)
5 4 3 2 1
6. Topic Management (topic relevance, topic coverage, and adequacy of details and
examples)
5 4 3 2 1
7. Confidence (anxiety-free speech)
5 4 3 2 1
8. Organization (initiation, development, termination / interconnectedness of ideas)
5 4 3 2 1
9. Strategy Use (avoiding unfamiliar language and compensating by using familiar language)
5 4 3 2 1
10. Time Management (timing your talk)
5 4 3 2 1
Total Score: -----------------------/50
Comments:------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------
Downloaded from ltj.sagepub.com at PENNSYLVANIA STATE UNIV on May 9, 2016
References (51)
Barkhuizen, G. (1998). Discovering learners' perceptions of ESL classroom teaching/learning activities in South African context. TESOL Quarterly, 32(1), 85-108.
Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151-167.
Boud, D. (2013). Enhancing learning through self-assessment. New York: Routledge.
Boud, D., & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment & Evaluation in Higher Education, 31(4), 399-413.
Butler, F. A., Eignor, D., Jones, S., McNamara, T., & Suomi, B. K. (2000). TOEFL 2000 speak- ing framework: A working paper (ETS Research Report No. RM-00-06). Princeton, NJ: Educational Testing Service.
Butler, Y. G., & Lee, J. (2006). On-task versus off-task self-assessment among Korean elemen- tary school students studying English. The Modern Language Journal, 90(4), 506-518.
Butler, Y. G., & Lee, J. (2010). The effects of self-assessment among young learners of English. Language Testing, 27(1), 5-31.
Chalhoub-Deville, M., & Wigglesworth, G. (2005). Rater judgment and English language speak- ing proficiency. World Englishes, 24(3), 383-391.
Chen, Y. M. (2006). Peer-and self-assessment for English oral performance: A study of reliability and learning benefits. English Teaching and Learning, 30(4), 1-22.
Chen, Y. M. (2008). Learning to self-assess oral performance in English: A longitudinal case study. Language Teaching Research, 12(2), 235-262.
Cheng, W., & Warren, M. (2005). Peer assessment of language proficiency. Language Testing, 22(1), 93-121.
Cohen, J. W. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). New York: Pearson.
Falchikov, N. (2013). Improving assessment through student involvement: Practical solutions for aiding learning in higher and further education. New York: Routledge.
Glaser, B., & Strauss, A. (1967). The discovery of ground theory: Strategies for qualitative research. Chicago, IL: Adline.
Handley, K., & Williams, L. (2011). From copying to learning: Using exemplars to engage stu- dents with assessment criteria and feedback. Assessment & Evaluation in Higher Education, 36(1), 95-108.
Hanrahan, S. J., & Isaacs, G. (2001). Assessing self-and peer-assessment: The students' views. Higher Education Research and Development, 20(1), 53-70.
Hendry, G. D., Bromberger, N., & Armstrong, S. (2011). Constructive guidance and feedback for learning: The usefulness of exemplars, marking sheets and different types of feedback in a first year law subject. Assessment & Evaluation in Higher Education, 36(1), 1-11.
Higgins, R., Hartley, P., & Skelton, A. (2002). The conscientious consumer: Reconsidering the role of assessment feedback in student learning. Studies in Higher Education, 27(1), 53-64.
Hughes, G. (2011). Towards a personal best: A case for introducing ipsative assessment in higher education. Studies in Higher Education, 36(3), 353-367.
Iwashita, N., Brown, A., McNamara, T., & O'Hagan, S. (2008). Assessed levels of second lan- guage speaking proficiency: How distinct? Applied Linguistics, 29(1), 24-49.
Jafarpur, A. (1991). Can naive EFL learners estimate their own proficiency? Evaluation and Research in Education, 5(3), 145-157.
Johnson, R. K. (1989). The second language curriculum. Cambridge: Cambridge University Press.
Kim, H. G. (2010). Investigating the construct validity of a speaking performance test. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 8, 1-30.
Kirby, N. F., & Downs, C. T. (2007). Self-assessment and the disadvantaged student: Potential for encouraging self-regulated learning? Assessment & Evaluation in Higher Education, 32(4), 475-494.
Kumaravadivelu, B. (1991). Language learning tasks: Teacher intention and learner interpretation. ELT Journal, 45(2), 98-107.
Kumaravadivelu, B. (1994). The postmethod condition: (E)merging strategies for second/foreign language teaching. TESOL Quarterly, 28(1), 27-48.
Kuo, I. (2011). Student perceptions of student interaction in a British EFL setting. ELT Journal, 65(3), 281-290.
Leger, D. S. (2009). Self-assessment of speaking skills and participation in a foreign language class. Foreign Language Annals, 42(1), 158-178.
Luoma, S. (2004). Assessing speaking. Cambridge: Cambridge University Press.
Luoma, S., & Tarnanen, M. (2003). Creating a self-rating instrument for second language writing: From idea to implementation. Language Testing, 20(4), 440-465.
Matsuno, S. (2009). Self-, peer-, and teacher-assessments in Japanese university EFL writing classrooms. Language Testing, 26(1), 75-100.
Mok, M. M. C., Lung, C. L., Cheng, D. P. W., Cheung, R. H. P., & Ng, M. L. (2006). Self- assessment in higher education: Experience in using a metacognitive approach in five case studies. Assessment & Evaluation in Higher Education, 31(4), 415-433.
Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218.
Orsmond, P. (2011). Self-and peer-assessment: Guidance on practice in the biosciences. Leeds: The Higher Education Academy Center for Biosciences.
Orsmond, P., & Merry, S. (2011). Feedback alignment: Effective and ineffective links between tutors' and students' understanding of coursework feedback. Assessment & Evaluation in Higher Education, 36(2), 125-136.
Orsmond, P., & Merry, S. (2013). The importance of self-assessment in students' use of tutors' feedback: A qualitative study of high and non-high achieving biology undergraduates. Assessment & Evaluation in Higher Education, 38(6), 737-753.
Orsmond, P., Merry, S., & Callaghan, A. (2013). Communities of practice and ways to learn- ing: Charting the progress of biology undergraduates. Studies in Higher Education, 38(6), 890-906.
Orsmond, P., Merry, S., and Reiling, K. (1997). A study in self-assessment: Tutor and students' perceptions of performance criteria. Assessment & Evaluation in Higher Education, 22(4), 357-368.
Orsmond, P., Merry, S., & Reiling, K. (2000). The use of student derived marking criteria in peer and self-assessment. Assessment & Evaluation in Higher Education, 25(1), 23-38.
Orsmond, P., Merry, S., & Reiling, K. (2002). The use of exemplars and formative feedback when using student derived marking criteria in peer and self-assessment. Assessment & Evaluation in Higher Education, 27(4), 309-323.
Patri, M. (2002). The influence of peer feedback on self-and peer-assessment of oral skills. Language Testing, 19(2), 109-131.
Plough, I. C., Briggs, S. L., & Van Bonn, S. (2010). A multi-method analysis of evaluation criteria used to assess the speaking proficiency of graduate student instructors. Language Testing, 27(2), 235-260.
Ross, J. A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment, Research, and Evaluation, 11(10), 1-13.
Rust, C., Price, M., & O'Donovan, B. (2003). Improving students' learning by developing their understanding of assessment criteria and processes. Assessment & Evaluation in Higher Education, 28(2), 147-164.
Saito, H. (2000). Peer, self, and instructor ratings of group presentations in EFL classrooms: A pilot study. The Journal of Rikkyo University Language Center, 2, 76-86.
Shimura, M. (2006). Peer-and instructor assessment of oral presentations in Japanese university EFL classrooms: A pilot study. Waseda Global Forum, 3, 99-107.
Stobart, G. (2008). Testing times: The uses and abuses of assessment. London: Routledge.
Tremblay, P. F., & Gardner, R. C. (1995). Expanding the motivation construct in language learn- ing. Modern Language Journal, 79(4), 505-520.
Wong, R. M. H. (2009). Try to describe the main point of your lesson: Student perception and identification of learning objectives in English lessons. Reflections on English Language Teaching, 8(2), 73-88.
Zhang, Y., & Elder, C. (2011). Judgments of oral proficiency by non-native and native English speaking teacher raters: Competing or complementary constructs? Language Testing, 28(1), 31-50.
Roya Pashmforoosh
Texas A&M University, Graduate Student
Papers
39
Followers
13
View all papers from
Roya Pashmforoosh
arrow_forward
Related papers
Speaking self-assessment: Mismatches between learners’ and teachers’ criteria
S. Taghaddomi
Esmat Babaii
Perceptual (mis)matches between teachers and learners are said to affect learning success or failure. Self-assessment, as a formative assessment tool, may, inter alia, be considered a means to minimize such mismatches. Therefore, the present study investigated the extent to which learners’ assessment of their own speaking performance, before and after their being provided with a list of agreed-upon scoring criteria followed by a practice session, matches that of their teachers. In so doing, 29 EFL learners and six EFL teachers served as participants; the learners were asked to assess their audio-recorded speaking performance before and after their being provided with the scoring criteria and practice session. The teachers were also asked to assess the learners’ performance according to the same criteria. Finally, the learners were required to evaluate the effectiveness of doing self-assessment in the form of reflection papers. The results revealed a significant difference between the learners’ assessment of their own speaking ability on the two occasions. The findings also suggested that providing the learners with the scoring criteria and the follow-up practice session minimized the existing mismatches between learner assessment and teacher assessment. Moreover, the inductive analysis of the reflection papers yielded a number of themes suggesting that, despite some limitations, the learners’ overall evaluation of the effectiveness of speaking self-assessment was positive.
Download free PDF
View PDF
chevron_right
Analysis of Self-Assessment Effect on EFL Students’ Speaking Performance
Era Adnyayanti
Journal of Educational Study, 2022
Studies indicated that EFL students were often struggling in speaking due to their fear of making mistake. This becomes a problem as speaking is an essential skill needed to be able to communicate in the target language and become successful learners. This study was conducted with the aim of investigating the effect of self-assessment on EFL students’ speaking performance. It used library research with a method of document analysis by analyzing previous related studies and theories. The results indicated that self-assessment has positive effect on students’ speaking performance as it involves students in the process of evaluating their learning process, product or performance, and progress. The effects include identification of strengths and weaknesses, improvement of self-efficacy, and improvement of speaking performance. However, self-assessment needs to be implemented well by considering the assessment criteria and practices in order to achieve these positive effects.
Download free PDF
View PDF
chevron_right
The Relationship between Self Peer and Teacher-Assessments of EFL Learners Speaking
RRP JOURNALS
Download free PDF
View PDF
chevron_right
The Effect of Self-Assessment on Iranian EFL Learners’ Speaking Skill
hamidreza fatemipour
International Journal of Applied Linguistics & English Literature, Austrailia, 2013
Self-assessment is one of the attractive new alternative assessment forms which raises learners' awareness of their individual progress, and encourages learners to become autonomous learners . The present study aimed to investigate the effect of self-assessment on speaking skill of Iranian EFL learners. The participants were 60 pre intermediate Iranian EFL learners. The participants in experimental group received training and practiced selfassessment of their speaking ability during the semester; however, participants in control group experienced no selfassessment. At the end of the semester, the participants in both groups were interviewed as posttest. The results revealed that self-assessment practices helped the participants of the study to improve their speaking ability. Moreover, participants had generally positive attitudes toward the notion of self-assessment and showed their willingness to continue self-assessment in future; however, they were in favor of teacher assessment too. The findings of the study offer several implications for teachers and practitioners.
Download free PDF
View PDF
chevron_right
Learners' Self-assessment of their Oral Performance in EFL Classrooms: The Case of Wollega University
rufael disasa
This thesis reports a study of conception and validity of learners' self-assessment of English oral performance and the effect of training learners in self-assessment on the validity and the conception in university EFL classrooms.
Download free PDF
View PDF
chevron_right
Relationships between Peer- and Self-Assessment and Teacher Assessment of Young EFL Learners’ Oral Presentations
Beth Samuelson
Educational Linguistics, 2016
As the traditional grammar translation approach is being gradually replaced by communicative or task-based approaches, paper-and-pencil tests, commonly used in English classes in Taiwan, do not meet the course goals. Alternative assessment, known for increasing learners' cognitive and meta-cognitive development as well as empowering students to take ownership of their learning, has been practiced extensively in L1 higher education, but neglected in L2 elementary schools. Thus, the purpose of this study is to investigate how peer and self-assessment can be implemented to evaluate young EFL learners' oral presentation and how the students perceive this experience. The study was conducted in two sixth grade classes at a public elementary school in southern Taiwan. After attending a professional development workshop held by the government, a local English teacher practiced peer and self-assessment in her class so as to engage every student in class activities and also to provide an opportunity for them to reflect upon their performance. In the process, the students formed groups of six to discuss and give grades after each individual student's oral report. Three types of data sources were analyzed. The first was the evaluation rubrics from peer groups, each presenting students, and the teacher. Then, a survey, containing 16 closed-format questions and one open-ended question, was administered to elicit the students' perceptions of the assessment process. Also, an interview was done with the teacher. The results show that peer and teacher assessment had strong positive correlation, whereas self-and teacher assessment were moderately correlated. The strength of correlation also varied for each evaluation criterion. Though learners responded positively to the assessing experiences in the questionnaires, they expressed concern that some grades assigned by peers were not fair and a few group members dominated the grading process. The findings shed light on benefits of combining peer and self-assessment and suggest training should emphasize self-assessment, evaluation criteria related to content of the presentation, and students' social skills to work harmoniously in groups. Most of all, students' traditional way of learning should not be neglected.
Download free PDF
View PDF
chevron_right
Authentic Assessment of Speaking Skill In EFL Class
Khaerunnisa Miraja
Siti Maysuroh
Journey: Journal of English Language and Pedagogy
The objectives of this study are to find out and determine the dominant type of authentic assessment used by teachers, the activity of authentic assessment implemented by teachers, the advantages and disadvantages of using authentic assessment in speaking class.This study used qualitative methods which employed interview, observation, and documents to collect the data. Using the purposive sampling technique, three English teachers at MA NW Perian under Yayasan Pondok Pesantren Pendidikan Darul Muttaqien (YP3DM) NWDI Perian were chosen as the participants of the study. The results of this research showed that (1) the dominant type of authentic assessment used in assessing speaking was performance based assessment; (2) the dominant authentic assessment activities used in assessing speaking were discussion, role-play and debate activity; (3) all of the English teachers agreed that using authentic assessment helped their students demonstrate and perform their true abilities without che...
Download free PDF
View PDF
chevron_right
Putting Rubrics to the Test: The Effect of Rubric-Referenced Peer Assessment on EFL Learners’ Evaluation of Speaking
Masoume Ahmadi
Journal of Academic and Applied Studies, 2013
This study attempted to shed some light on the effect of rubric-referenced peer assessment on EFL learners' speaking skill and on the cultivating the learners' awareness of having appropriate criteria for speaking, as one of the four major skills. This study explored the effect of rubrics on peer assessment of 18 Iranian EFL learners. First, learners assessed their classmates speaking performance based on their own presuppositions and assumptions. Subsequently, a spoken language rubric was introduced to them. They reassessed their classmates' performances through using this rubric. Quantitative data analysis revealed significant difference between the results. In-depth qualitative analyses of comments and marginal notes written down by learners revealed that peers heed not only to institutional components specified in scoring scales but also to other irrelevant criteria such as the result of the speech act performed.The study has suggested that the use of a combination of peer assessment and rubric-referenced assessment encourages students to become more rationally responsible and reflective and has shown positive formative effects on student achievement and attitudes. The article concludes with some guidelines for practitioners.The findings of this study also provide insight into the effective assessment and recommendations for future research and practice are made.
Download free PDF
View PDF
chevron_right
Exploring the perceptions of English second language teachers about learner self-assessment in the secondary school
lebohang moloi
2009
The main purpose of this study was to investigate English second language (ESL) teachers' perceptions about learner self-assessment in the secondary school. The study also examined the factors that might influence the perceptions that teachers hold about self-assessment. The participants of this study were 163 ESL teachers who are teaching at secondary schools in Gauteng Province; they were from 94 randomly selected schools. The data was collected through questionnaires and the response rate was approximately 53%. The literature review was utilized to identify the main perceptions. The studies led to a 57-item teacher's perceptions of learner self-assessment questionnaire based on the following main perceptions: the value of selfassessment in the teaching and learning processes, the use of selfassessment as an alternative assessment technique in English second language learning, the accuracy and reliability of self-assessment and the role of teachers in the effective implementation of self-assessment in their classrooms. The data for the study were analysed using quantitative techniques. The findings for the study showed that teachers have a strong feeling about the value of learner self-assessment; they showed that it helps in the improvement of teaching and learning. The results also indicated that teachers agree that self-assessment should be used in English second language teaching as an alternative form of assessment as it enhances learners' linguistic skills. Teachers perceive that they have an important role to play in ensuring that the implementation of self-assessment becomes successful in their classrooms. However, some have indicated that learners are not objective enough when assessing themselves and therefore the results derived from self-assessment cannot be accurate and reliable. Teachers' perceptions correspond with their classroom self-assessment practice because most of them also indicated that they employ self-assessment to enhance learners' performance. Majority of teachers also indicated that factors such as class size, availability of materials, teacher professional training and development, formal teacher training, years of experience, classroom experience with self-assessment, and time available for self-assessment played a significant role in influencing their perceptions. The conclusions were drawn from the results and the recommendations were made for further research. The limitations of the study were also discussed.
Download free PDF
View PDF
chevron_right
Current Problems and Proposed Solutions for Testing Speaking: Opinions of EFL Teachers
Dr. Devrim HOL
This paper is an attempt to explore EFL teachers’ opinions about the problems they encounter and proposals for solutions with regard to the speaking test implemented at a state university in Turkey. The participants were 22 EFL instructors teaching English at Pamukkale University in Turkey. The data were gathered through a semi-structured interview form developed by the researcher. The form was developed based on the review of literature. Among the content analysis methods, categorical analysis method and inductive analysis method were used to analyze the data obtained through semi-structured interview form administered to the teachers. The results were analyzed and discussed in terms of the speaking test and its implications for teaching and learning activities and testing. In the light of the study findings, majority of the teachers thought that time limitation is a factor that affects the validity and reliability of speaking test. In addition, teachers stated that formative assessment during the academic year would produce better results performances in testing speaking.
Download free PDF
View PDF
chevron_right
Cited by
The Effects of Task-Based Speaking Tasks on Iranian University Students Willingness to Communicate and Self-Efficacy in Online Classes: A Mixed-Methods Study
Siamak Rahimi
Education Research International, 2022
This mixed-methods study attempts to explore the effects of TBLT on the development of Iranian university students' self-efficacy and willingness to communicate in speaking skills. To this end, two intact classes were selected in Ayatollah Borujerdi University in Borujerd City, Iran. The experimental group went through an eight-week teaching program, whereas the control group received the usual content-based instruction. The data were gathered from multiple sources at various time points using two questionnaires and a focus group interview. The results of the quantitative analysis using ANCOVA revealed a positive effect of the online TBLT approach on university students' self-efficacy and willingness to communicate in speaking in the Iranian context. Moreover, the findings of the focus group interview disclosed several themes about the merits of TBLT on self-efficacy and willingness to communicate in oral interaction skills. In the end, a range of implications is suggested f...
Download free PDF
View PDF
chevron_right
The longitudinal development of self-assessment and academic writing: an advanced writing programme
Anna Zólyomi, Ph.D.
Language Learning in Higher Education
Although several studies have investigated the self-assessment (SA) of writing skills, most research has adopted a cross-sectional research design. Consequently, our knowledge about the longitudinal development of SA is limited. This study investigated whether SA instruction leads to improvement in SA accuracy and in second language (L2) writing. A total of 33 English as a foreign language (EFL) students composed and self-assessed two argumentative essays, one at the beginning (Time 1) and one at the end (Time 2) of a semester-long advanced writing (AW) programme at a Hungarian university. About half of the participants received SA instruction (experimental group), while the other half did not (control group). The essays were scored by two teachers and analysed for linguistic complexity. The results showed improvements in SA accuracy in both groups. However, the SA-teacher assessment (TA) correlation for the total score was statistically significant only in the experimental group at...
Download free PDF
View PDF
chevron_right
A Case Study on the Effect of Self-Assessment on EFL Learners’ Self-Efficacy Beliefs
Dilara Demirbulak
European Journal of English Language Studies
This study aimed to look into the impact of continuous self-assessment on the self-efficacy beliefs of undergraduates studying English as a foreign language (EFL). To find out if there is a change in students’ beliefs about their language proficiency perception, the research was undertaken with 102 participants at the school of foreign languages of a private university. It was designed as quasi-experimental research utilizing qualitative and quantitative research methods to collect data through a self-efficacy scale, self-assessment questionnaire, and self-reflection checklists. The control and experimental groups were administered a self-efficacy scale at the beginning and the end of the term. Throughout the term, the participants in the experimental group assessed themselves weekly through self-reflection checklists and a self-assessment questionnaire. Based on the overall results, it was found that when the students had an opportunity to identify their strengths and weaknesses du...
Download free PDF
View PDF
chevron_right
Peer assessment in group-oriented classroom contexts: on the effectiveness of peer assessment coupled with scaffolding and group work on speaking skills and vocabulary learning
Mina Homayouni
Language Testing in Asia
Peer learning, also known as collaborative learning, is based on social constructivism and contends that learning takes place more actively when students interact socially with their peers. This study sought to examine the effects of scaffolded peer assessment in group-oriented classrooms on developing speaking skills and enlarging the vocabulary size of language learners. To accomplish this objective, through cluster sampling, the researchers selected 37 lower-intermediate and 5 intermediate learners of English. Then, 20 lower-intermediate subjects were randomly assigned as the experimental group along with the 5 more proficient learners. In groups of 5, the intermediate learner was assigned the role of the mediator and was responsible for giving feedback to their peers. The rest of the subjects were assigned as the control group and there was no mediator in their group. Four instructional sessions were allocated to scaffolded peer assessment of speaking and four sessions were devo...
Download free PDF
View PDF
chevron_right
Self vs. peer assessment activities in EFL-speaking classes: impacts on students’ self-regulated learning, critical thinking, and problem-solving skills
Dr. Seyyed Mohammad Ali Soozandehfar
Language Testing in Asia
Assessment has several advantages like involving students in learning and making them cognizant of their strengths and weaknesses. The effects of peer assessment and self-assessment as two main kinds of assessment have not examined on EFL learners’ self-regulated learning, critical thinking, and problem-solving skills; therefore, the present research attempted to inspect the impacts of self and peer assessment activities on self-regulated learning, critical thinking, and problem-solving skills of Iranian EFL learners. To do so, 75 Iranian EFL learners were selected and divided into two experimental groups (peer assessment group (PAG) and self-assessment group (SAG)) and one control group (CG). Then, three questionnaires of self-regulated learning, critical thinking, and problem-solving skills were administered to the students as the pre-tests of the study. After that, one experimental group received the treatment via using self-assessment and the other experimental group received th...
Download free PDF
View PDF
chevron_right
Explore
Papers
Topics
Features
Mentions
Analytics
PDF Packages
Advanced Search
Search Alerts
Journals
Academia.edu Journals
My submissions
Reviewer Hub
Why publish with us
Testimonials
Company
About
Careers
Press
Content Policy
580 California St., Suite 400
San Francisco, CA, 94104