Promoting Metacognitive Awareness in Writing Assessment Tasks through Planning, Monitoring, and Evaluation: Achievements and Perceptions (Research Paper)

Promoting Metacognitive Awareness in Writing Assessment Tasks through Planning, Monitoring, and Evaluation: Achievements and Perceptions

^[1]Mojgan Kouhpayehzadeh Esfahani

^[2]Mojgan Rashtchi*

^[3]Ali Asghar Rostami Abousaidi

[4]Bahram Mowlaie

Research Paper                                                     IJEAP- 2209-1902        DOR: 20.1001.1.24763187.2022.11.3.4.9

Received: 2022-09-02                                      Accepted: 2022-10-15                                     Published: 2022-10-29

Abstract

The present study followed a sequential explanatory mixed-methods design in two phases to investigate the effects of cognitive and metacognitive strategies on developing Iranian EFL learners’ writing assessment performance. For the quantitative stage, the researchers conducted a quasi-experimental study with a non-equivalent pretest-post-test control group design. To this aim, 41 upper-intermediate EFL university students were selected from two intact classes based on convenience sampling from the Islamic Azad University, South Tehran Branch. The classes were randomly assigned to a Cognitive-based Assessment Instruction Group (CAIG) and a Metacognitive-based Assessment Instruction Group (MAIG). A general proficiency test administered before the treatment verified the participants’ homogeneity. The Metacognitive Awareness Writing Questionnaire was administered before and after the intervention to measure the participants’ metacognitive awareness levels and self-regulation. MAIG used metacognitive strategies such as planning, monitoring, and revising to assess English essays. CAIG performed assessments based on cognitive strategies and a Writing Rubric. After the treatment, the participants took two writing posttests on rehearsed and unrehearsed topics. Results of the Mann-Whitney U test and Multivariate Analysis of Variance (MANOVA) indicated that MAIG outperformed CAIG in both posttests. Raising metacognitive awareness regarding self-regulation and metacognitive strategies could significantly enhance the participants’ writing assessment performance. In the final stage, online semi-structured interviews were performed, the findings of which were strengthened through content analysis. The study has implications for test and materials developers, EFL teachers and students, and syllabus designers in expanding their understanding of raising metacognitive awareness and its application in writing assessment.

 Keywords: Cognitive Strategies, Metacognitive Awareness, Metacognitive Strategies, Writing Assessment Tasks

1. Introduction

Trends in the testing literature show that assessment is synonymous with giving grades to measure learning. Generally, grades act as a type of reward for completing a task or a punishment for not completing it. In reality, however, teachers believe that this is not the ideal and primary purpose of testing. Typically, assessment is a subjective judgment, and ELT teachers’ and students’ lack of assessment literacy poses challenges for both of them.

Many educational systems prioritize summative or traditional assessment, which refers to the assessment of learning above formative assessment or assessment for learning. Whereas the teacher has a prominent role in summative assessment, the position of students along with their teachers and peers, is neglected in formative assessment. However, the student-centered approach, also known as assessment as learning, puts assessment on the student’s shoulders (Santos & Semana, 2015). In this regard, assessment responsibility gradually shifts from teacher-centeredness to student-centeredness to facilitate learning assessment and self-regulation. In assessment, students are actively involved in goal setting or planning, monitoring their improvement, and making decisions on how to fill in any learning gaps.

This study, in line with Vygotsky’s (1978) Sociocultural Theory (SCT), applied the theory within the classroom to help learners become interactive, autonomous, and responsible. The SCT aspects place learners at the center of the learning process and encourage them to regulate, control, and improve problem-solving skills. This study attempted to offer how a student-centered approach could be used in the L2 writing classroom and contribute to writing assessment.

A vast majority of empirical research has concentrated on traditional techniques for assessment, and little attention has been paid to alternative approaches, such as strengthening metacognitive awareness for learning strategy improvement, particularly the writing skill (Xiao, 2007). Within the SCT framework, in the present study, the teacher decided to transfer the writing assessment responsibilities to the learners through scaffolding, discussion, self-questioning, goal setting, thinking aloud, and cooperative learning, which resulted in better decisions, problem-solving, critical thinking, self-regulation, and self-efficacy. This ultimately led them to an automatic and persistent state without external support. It is beneficial for teachers to enhance communicative-oriented, group interaction, and metacognitive knowledge awareness-raising in their classroom setting. By considering metacognitive awareness as a part of writing assessment, learners realize that assessment is a deliberate, thoughtful process leading them to learn the characteristics that might affect their final scores.

2. Literature Review

A number of scholars (e.g., Dinsmore & Alexander, 2012) proposed strategy taxonomy as including both surface and deep processing. In the surface learning process, learners are not quite strategic, such as unskilled readers or writers; they use fewer strategies, mainly less complex ones. On the other hand, skilled readers or writers process a well-developed repertoire of strategies that help them successfully realize performing the task and eventually achieve the cognitive goal. This taxonomy can be implied as cognitive and metacognitive strategies in the learning process. The purpose of cognitive strategies is getting through the obstacles encountered along the way, with functions that are narrower in scope (Xiao, 2007). However, in the deep processing or metacognitive strategic learning process, learners need to focus on communicative purposes, critical thinking, and problem-solving activities. Cognitive strategies are applied as supplementary strategies that aid in implementing metacognitive strategies (Dinsmore & Alexander, 2012).

Cognitive strategies are mainly restricted to performing explicit writing assessment tasks, focusing on the outcome or results, as well as whatever can be done to ensure the assessment precisely. On the other hand, incorporating metacognitive strategies in teaching writing assessment is extremely important because learners tend to become immersed in the assessment task and constantly monitor their progress. Implementing metacognitive strategies helps learners shift from passive to active learners engaged in the assessment process and be aware of improving their learning assessment. When learners get engaged in metacognitive strategies, they employ monitoring, analyzing, and evaluating, which belong to higher-order thinking skills (Rashtchi & Khoshnevisan, 2020). Thus, they can adopt strategies to perform the assessment task. Proper training in writing assessment will help learners develop inner criteria for accurate judgment in evaluating writing tasks and become more successful, even in their future professions (Xiao, 2007).

A systematic literature review shows that no research has investigated the relationship between metacognition and the attainment of EFL students in assessing writing. Accordingly, this study aimed to highlight the utility of metacognitive awareness-raising in writing assessment. Since EFL learners’ metacognitive awareness in assessing writing is still an area of research that has been investigated to a small extent, it is significant to develop a scientific discussion in this field.

In 2021, Oudman, van de Pol, and van Gog investigated whether self-assessment could foster school learners’ regulation and monitoring accuracy. Their findings showed that learners’ monitoring and evaluation are related, and self-assessment can be a beneficial tool to improve learners’ monitoring and evaluation accuracy. In this vein, Zhao and Liao (2021) examined the effect of metacognitive writing strategies among 200 EFL students and discovered the limited and mixed effects of the metacognitive strategies use in writing assessment. In addition, McMillan and Hearn (2008) examined the impact of metacognitive strategies such as self-regulation and self-monitoring on learners’ autonomy, self-efficacy, and confidence development. As a dynamic process, they found that self-assessment enforces students to self-monitor, self-evaluate, and recognize corrections to learn.

In Iran, various studies have focused on the role of metacognition in different aspects of EFL learning; however, the effects of metacognitive strategies and writers’ thinking processes on their writing assessment have almost been overlooked. For example, Khodabakhsh, Abbasian, and Rashtchi (2018) showed the positive effects of dynamic assessment models in developing language awareness and metacognitive strategy use in second-language writing classes. Nemati and his colleagues (2021) examined learners’ metacognitive awareness to develop EFL learners’ writing assessments. Nemati, Ghafoori, Birjandi, and Izadpanah (2021) explored the effect of various assessment types, such as self, peer, and teacher assessment, on the development of Iranian EFL learners’ writing assessment. They revealed the importance of cognitive and metacognitive strategies in dynamic assessment types.

The current research focused on the effects of raising metacognitive awareness on improving writing assessment performance. This study is distinctive and novel in that no national research has yet been undertaken on the role of the metacognitive approach in L2 writing assessment. The bulk of studies of EFL research have investigated how metacognition awareness-raising impacts writing ability (e.g., Beiki, Raissi, & Gharagozloo, 2020; Teng, Qin, & Wang, 2021). However, after obtaining the metacognitive approach instruction, learners’ writing assessment performance was rarely addressed. Thus, the significance of the current study lies in the novelty of its examining the efficiency of teaching metacognition and evaluating EFL students’ attainment in assessing essays. Furthermore, it helps broaden EFL learners’ comprehension of the advantages of increasing metacognitive awareness.

The primary purpose of the current study was to probe the effect of cognitive and metacognitive strategies on improving EFL Iranian learners’ writing assessment, which might help them develop their awareness of how these strategies would affect their assessment of writing. In the current research, the researchers focused on the significant contributions of metacognitive awareness and writing assessment based on Flavell’s (1979) theoretical two-dimensional metacognition framework, including metacognitive knowledge and metacognitive regulation. Metacognitive knowledge refers to what one knows about cognition, whereas regulation of cognition means how individuals apply that knowledge to regulate their knowledge (Flavell, 1979). In this context, this research intends to answer the following research questions:

Research Question One: To what extent is writing assessment affected by the instruction of metacognitive-based or cognitive-based strategy instructions?

Research Question Two: To what extent do metacognitive-based assessment instruction and cognitive-based assessment instruction result in EFL learners’ self-regulation? 

Research Question Three: Do metacognitive-based instruction and cognitive-based instruction participants perform differently in assessing essays?

Research Question Four: What are EFL learners’ perceptions toward metacognitive-based writing assessment instruction?

3. Method
   • Research Design

The present study followed a sequential explanatory mixed methods design in two phases. For the quantitative phase, the researchers conducted a quasi-experimental study with a non-equivalent pretest-posttest control group design to investigate the effects of cognitive and metacognitive awareness on developing Iranian EFL learners’ assessment ability. In the final stage of the qualitative phase, semi-structured interviews were performed, helping the researchers control bias and insufficiencies of the collected data, and providing more consistency and depth to the findings (Creswell, 2021).

• Participants
  • EFL Learners

Forty-one Iranian upper-intermediate EFL university students in two intact classes were selected based on convenience sampling from the Islamic Azad University (IAU), South Tehran Branch. The participants were females and males whose ages ranged from 22 to 30 years old and were master’s students majoring in TEFL. The writing courses for this research were developed with the university English Language Department’s aid, where the study was conducted. The educational program randomly assigned them to two intact classes in advance; one class was randomly allocated to the Cognitive-based Assessment Instructional Group (CAIG, n=20) and the other to the Metacognitive-based Assessment Instructional Group (MAIG, n=21). An English Proficiency Test (EPT) controlled the participants’ proficiency levels, and only those with one standard deviation above or below the mean were selected. As verified by the EPT results, the participants’ language proficiency was upper-intermediate or B2 (CEFR). Participants took part in an EFL writing course held one session each week for 14 sessions. The treatment took 60 minutes a session in both classes. Because of the COVID-19 outbreak, one of the researchers taught the course online.

• Raters

Two knowledgeable instructors then reevaluated the participants’ rated essays. They had been trained to score the sample papers developed by the CEFR (2019) based on a writing rubric (WR) to ensure inter-rater reliability between the scores (0.86) and a high consistency between them.

• Instruments

The quantitative data were collected by the EPT, Metacognitive Awareness Writing Questionnaire (MAWQ), Writing Assessment Pretest and Posttests, and a writing rubric in two research groups (MAIG and CAIG). The information assortment was painstakingly directed, and then data were analyzed by the SPSS software version 26 (2019). An English Proficiency Test (EPT) adopted from the Cambridge Preliminary English Test (PET) (2016) was used to examine the participants’ homogeneity. The test had 42 items in eight subsections and assessed reading comprehension and writing skills. The test’s reliability, as determined by KR-21, was 0.75. The MAWQ was used before and after the treatment for the research groups to assess EFL learners’ metacognitive awareness levels. Farahian’s (2017) MAWQ is based on Flavell’s (1979) two-dimensional metacognitive model. The MAWQ assesses cognitive and regulatory knowledge. On a five-point Likert scale, it has 36 statements with nine subcategories (accessible in Farahian, 2017). The MAWQ is used to measure self-regulation knowledge, including declarative knowledge (task knowledge), declarative knowledge (person knowledge), procedural knowledge, and conditional knowledge. It also assesses metacognitive regulatory strategies, which are divided into five categories: (a) planning (goal setting), (b) monitoring (learning and strategy assessment), (c) general strategies, (d) evaluation (performance and strategy effectiveness analysis), and (e) revision. Confirmatory Factor Analysis was used to examine the construct validity of the knowledge of cognition in the MAWQ. Farahian evaluated the questionnaire’s reliability to be 0.78 using Cronbach’s alpha (2017).

All participants assessed three essays written by some EFL learners in another university to enable the researchers to measure their writing assessment before and after the treatment. One was employed as the pretest, while two other writing assessments were used as posttests. One posttest, identical to the pretest, asked, “Describe a noteworthy experience in your life,” and the other posttest was on an unrehearsed topic, “Describe your role model,” to account for the practice effect and to measure the participants’ writing evaluation skills after a time delay. The essay topics were chosen from Objective First of Cambridge University Press, the participants’ textbook (Capel & Sharp, 2014).

The objective writing assessment rubric from Cambridge English Language Assessment (2019) was utilized to compare the participants’ and raters’ assessments. The Writing Rubric (WR) has four subscales: content, communicative achievement, organization, and language, and responses were graded from 1 to 5 on each subscale (Appendix A).

A semi-structured interview with preplanned questions was performed to triangulate the data. The tool served as a self-report for the participants to show to what extent they concentrated on writing assessment in both metacognitive and cognitive instruction strategies. The researchers interviewed the participants to reflect on their use of metacognition strategies as they assessed the essays based on the training they received (Appendix B).

• Quantitative Data Collection Procedure

The research groups performed two different assessment instructions: CAIG received cognitive-based assessment training, whereas MAIG received metacognitive awareness-raising instruction. The participants did not know they were taking part in a study to mitigate the Hawthorne effect.  

• Pretest

In the first week, the teacher administered the EPT to ensure that the participants were homogeneous. The second week was allotted to delivering the MAWQ (Farahian, 2017). In the next session, the teacher familiarized the group members with the different dimensions of WR. In the subsequent session, all the participants of the two groups rated a composition “Describe a noteworthy experience in your life” to enable the researchers to examine their ability in writing assessment and allow them to work with the WR practically.

• Implementation of Cognitive-based Assessment Instruction (CAI)

The CAI aims to emphasize fair judgment and achieve a higher score instead of focusing on increasing awareness and improving knowledge. The instructor in CAI provided some cognitive strategies like mind mapping to determine learners’ goals without discussion or articulation of assessment. After administering the pretest in the CAIG from sessions five to twelve, the researchers followed the explicit cognitive processes involved in the assessment and the directions to score based on the scoring writing rubric (WR). The WR guidelines for assessment and judgment were considered the benchmark for interpreting learners’ assessment levels. In CAIG, the researchers followed three steps in the following sessions after presenting the pretests:

Step one (Prewriting assessment): In the fifth session, the instructor covered prewriting assessment procedures such as goal setting and preparing to achieve a goal. The teacher guided the participants via brainstorming, memory, noticing, and decision-making to find and arrange various ideas in the essay and make a plan. The essay was then scored using the WR criteria, with an emphasis on planning. By activating their previous knowledge and mind mapping, the researchers prepared and assisted the learners in identifying what they knew about assessment. During the sixth and seventh sessions, the participants had to decide which idea stated in the essay was the most exciting and relevant to the topic. They assessed the structure of the essay, paying particular attention to the topic sentence, main idea, body paragraphs, and conclusion. Because of this, the instructor in the sixth session initially helped the students expand their comprehension of writing assessment knowledge, which detailed language regulations, by reminding them to take into account linguistic knowledge, structure, writing purpose, theme, and even essay layout.

Step two (Note-taking): The instructor specified four objectives for practicing writing assessment in sessions seven and eight: organization, unity, support, and error-free sentences. The students learned how to jot down or take notes on whatever the instructor taught them in this lesson. The participants used WR to evaluate the first offered essay while keeping the four aims in mind. They also learned how to utilize their thoughts to think and anticipate the assessment’s beginning or end.

Step three (Revising and Reflecting): In sessions nine and ten, the participants concentrated on reviewing and revising the essay’s structure and content. They reviewed the texts, read them silently or aloud, and considered if they could add additional detail or remove any unrelated ideas. Before the final score, the learners might proceed to the next step and revise the text’s structure and content. The learners worked on revising the essay’s structure and organization throughout the tenth and eleventh sessions. In the revising stage, overhauling the structure and arrangement of paragraphs is a crucial component of the writing assessment.

3.4.3. Implementation of Metacognitive-based Assessment Instruction (MAI)

Training in this group began with the fifth session, just as CAIG. The teacher consciously and actively used metacognitive-based assessment instructions to raise the students’ awareness so that students could gauge their assessment as informational rather than judgmental. They exercised meaningful assessment by thinking critically, monitoring, analyzing, and developing solutions. Internal attributions deriving from their ability and mental effort or awareness were addressed rather than external attributions such as peer support, as in CIAG. The following demonstrates how the teacher utilized guided practice and modeling to assist participants in choosing, monitoring, and evaluating in assessment. Flavell’s (1979) two-dimensional metacognition framework, comprising metacognitive knowledge and metacognitive control, was emphasized in this instruction.

3.4.3.1. Knowledge of Cognition (Metacognitive Knowledge)

Step one (Preparation): The fifth session stressed building learners’ cognition awareness, which could disclose what learners understood about their cognition process as well as knowledge to regulate their cognition process. Flavell (1979) divides these knowledge variables into three categories: person knowledge (declarative knowledge), task knowledge (procedural knowledge), and strategic knowledge (conditional knowledge). In improving declarative knowledge, the instructor first raised the learners’ awareness and thoughts about themselves as thinkers via activating the participants’ background knowledge as well as the elements that impact their writing assessment and prepared them to plan how to tackle the assessment. Within this context, the teacher addressed the structure of an essay to teach the students “how” to assess written tasks by alerting them to examine linguistic skills, structure, the goal of writing, theme, and even layout.

Learners with a high level of procedural knowledge complete tasks more automatically, have a larger repertoire of strategies, sequence strategies effectively, and employ multiple strategies to solve issues (Lee & Mak, 2018). Finally, the teacher explained to the participants “when” and “why” strategies were appropriate for solving the problem and strengthening conditional knowledge.

In the subsequent sessions of the MAI, the teacher concentrated on metacognitive regulations, which refer to processes that learners use to regulate their cognitive activities and be responsible for carrying out a writing assessment in terms of three main strategies: planning, monitoring, and evaluating.

3.4.3.2. Regulation of Metacognition (Knowledge of Regulation)

Step two (Presentation & Practice): In this step, the participants received instruction and explanation on the regulation of cognition through planning, monitoring, and evaluation strategies related to all kinds of learning tasks that can develop learners’ metacognitive awareness (Oudman et al., 2021; Teng et al., 2021). This process entails two main functions: determining what they need to learn and how they can learn it. Likewise, regulatory skills, cognitive control, and self-regulation also point to the three metacognitive strategies’ functions. In this aspect, metacognitive strategies are the decisions learners make before (planning), during (monitoring), and after (revising) the learning procedure.

Planning: In sessions six and seven, the teacher selected planning strategies to involve the participants in problem-solving steps. She focused on learners’ ideas and suggestions before performing the assessment task. To plan or determine how to assess, they needed a variety of knowledge, including world knowledge, rhetorical understanding, and linguistic knowledge. The major goal of the assessment was to encourage the students to contribute ideas through goal setting, discussion, self-questioning, pondering aloud, and cooperative learning (Teng et al., 2021; Zhao & Liao, 2021). Learning goals and preplanning helped the participants correct their misconceptions and identify their strengths and limitations. Time restrictions, purposes, and guidelines related to the learning process were taught explicitly by the teacher.

Monitoring: In sessions eight and nine, the monitoring procedure was introduced by the instructor as one of the strategies for improving metacognitive awareness. She described how to pay attention to grammatical, linguistic, and contextual problems, distinguish irrelevant information, and identify semantic ambiguity. When monitoring and assessing, students received the teacher’s and peers’ comments and feedback on the decisions. Higher-order or metacognitive procedures for choosing and monitoring assessment judgments encourage creative, critical thinking, and meaningful learning (Rashtchi & Khoshnevisan, 2020). The monitoring strategy assists the learners in igniting and refining their decisions via self-questioning.

The instructor also proposed a model for monitoring their evaluation. She requested the learners to explicitly describe how they were spotting the errors in the compositions while monitoring them. They determined whether any adjustments were required throughout the reviewing phase to assist the text in becoming stronger. The participants then scored the essay using WR. Successful learners can demonstrate their learning and thinking processes, establish realistic learning goals, monitor their progress toward achieving those goals, and organize their assessment by dealing with errors meaningfully.

Evaluating: In the 10^th and 11^th sessions, the instructor asked the learners to evaluate and revise a selected essay. Evaluation involves assessing and appraising the final products and effective learning. A few researchers mention that metacognitive knowledge and regulatory skills such as planning are associated with evaluation (Teng et al., 2021). During the assessment task, learners’ decision-making was essential to implementing their rhetorical plan and revising it. Also, they could increase their proficiency and experience in realizing their potential to improve their problem-solving and logical reasoning. The assessment tasks were completed by the learners through debriefing, discussion, peer and instructor feedback, and self-questioning. The emphasis in this procedure was on practicing fairness in assessment. As a result, learners eventually learned self-regulation procedures and relied more on themselves than their teacher.

• Posttest

During the 12th to 14th sessions, the MAIG and CAIG participants assessed two rehearsed and unrehearsed essays based on the cognitive strategies via WR. They also completed MAWQ. At the final stage, an online semi-structured interview was performed with 12 volunteer participants from MAIG.

3.5. Qualitative Data Collection Procedure

The semi-structured interviews were conducted at IAU, the South Tehran Branch, English Language Department. Due to the outbreak of COVID-19, the online interview with 12 EFL learners (seven females and five males) who volunteered from MAIG were virtual. The in-depth interviews enabled the researchers to explore the participants’ perceptions more deeply regarding the instruction (Creswell, 2021). The semi-structured interviews encompassed 11 questions (Appendix B), and the researchers transcribed the recorded interview files with the participants’ permission based on the following steps. First, they reviewed the files to get a conceptual understanding of the data. They then implemented a coding process to identify common themes, organize them, and decide if further data was needed (Creswell, 2021). In this process, they divided the texts into segments, coded them, extracted the resulting codes into the required ones, and finally reduced them to 4-6 themes.

4. Results
   • Exploring the First and Second Research Questions

The first and second research questions addressed the effect of metacognitive-based and cognitive-based strategies instructions on the writing assessment and self-regulation of the study groups. Table 1 shows that the normality assumption was retained in CAIG and MAIG.

The researchers ran two MANOVA’s to compare the MAIG’s and CAIG’s means on the pretests of self-regulation and metacognitive strategies. Then they compared their means on the posttests of self-regulation and metacognitive strategies. The first MANOVA examined whether the two groups were homogenous regarding self-regulation and metacognitive strategies, while the second MANOVA explored the first and second research questions. The results are discussed below.

4.1.1. Self-Regulation and Metacognitive Strategies, Pretest

A MANOVA was conducted to compare the CAIG’s and MAIG’s means on the pretests of self-regulation strategies and metacognitive strategies to examine whether the two groups were homogenous regarding self-regulation strategies and metacognitive strategies before the treatment. Besides the assumption of normality reported above, MANOVA has two more assumptions: homogeneity of covariance matrices and homogeneity of variances. The non-significant results of the Box’s test (Box’s M=7.88, p>.001) indicated that the assumption of homogeneity of covariance matrices was met. That is to say, the correlations between pretests of self-regulation strategies and metacognitive strategies were roughly equal across the two groups. As noted by Filed (2018), the results of the Box’s test should be reported at .001 levels.

The results of MANOVA (F (2, 38) = .771, p > .05, partial η² = .039 representing a weak effect size), as indicated in Table 3, reveals no significant differences between the CAIG’s and MAIG’s overall means on the pretests of self-regulation and metacognitive before the treatment.

Based on the descriptive statistics (Table 4) and the between-subject effects (Table 5), it can be concluded: A: There is no significant difference between the pretest mean scores of MAIG (M=3.22) and CAIG (M=3.24) in self-regulation (F (1, 39) =.029, p >.05). B: There is no significant difference between MAIG’s (M = 2.96) and CAIG’s (M = 3.10) means on the pretest of metacognitive strategies (F (1, 39) = 1.40, p>.05).

Table 5: Tests of Between-Subjects Effects; Pretests of Self-Regulation and Metacognitive Strategies

4.1.2. Self-Regulation and Metacognitive Strategies, Posttests

To probe the first research question, the researchers conducted a MANOVA to compare the CAIG’s and MAIG’s means on the posttests of self-regulation strategies and metacognitive strategies. The non-significant results of the Box’s test (Box’s M=13.36, p > .001) indicated that the assumption of homogeneity of covariance matrices was retained. In other words, the correlations between posttests of self-regulation strategies and metacognitive strategies were roughly equal across the two groups.

The MANOVA results (F (2, 38) = 16.69, p < .05, partial η² = .468, representing a large effect size) indicated significant differences between the CAIG’s and MAIG’s overall means on the posttests of self-regulation and metacognitive strategies (Table 7). The findings presented in Tables 7 and 8 enabled the researchers to conclude that there was no statistically significant difference between the self-regulation and metacognitive strategies of the study groups.

Table 8: Tests of Between-Subjects Effects; Posttests of Self-Regulation and Metacognitive Strategies

• Exploring the Third Research Question

A Mann-Whitney U test was performed to compare the CAIG’s and MAIG’s mean ranks on the pretest to examine whether the groups were homogenous regarding their writing assessment ability before the treatment. The results showed that CAIG (Mdn=16.50) and MAIG (Mdn = 17.00) had very close medians. Table 9 reveals the results of the Mann-Whitney U test (Z =-.396, p>.05), indicating no significant differences between the two groups before the treatment.

After the treatment and to measure the participants’ writing assessment before and after the treatment, a MANOVA was performed to compare CAIG’s and MAIG’s means on the rehearsed and unrehearsed writing assessment posttests. The non-significant results of the Box’s test (Box’s M=11.72, p>.001) indicated that the assumption of homogeneity of covariance matrices was retained.

The results of between-subjects-effects, as indicated in Table 11, urges the researchers to conclude that: The MAIG (M=11.80) significantly outperformed CAIG (M=15.10) on the rehearsed and unrehearsed writing assessment posttests (F (1.39) =57.65, p<.01, partial η² =.596 representing a large effect size). MAIG’s assessment conforms to the raters’ mean assessment (10), while the CAIG’s assessment is significantly different from the two raters’ ratings.

Table 12 shows the descriptive statistics of the groups: MAIG (M=15.14) and CAIG (M=18.00) on the rehearsed writing assessment posttest. MAIG’s assessment is lower than CAIG and conforms to the raters’ mean assessment (14.5). CAIG’s assessment is significantly different from the two raters’ ratings. This finding shows that MAIG gave lower scores to the writings and was more accurate in assessing (as the mean was closer to those of the raters).

Recent research has revealed that the EFL learners in CAIG mostly focused on writing mechanics such as spelling, grammar, vocabulary, and punctuation, while MAIG learners concentrated on organizational, contextual factors, and stylistic writing, which revealed the importance of variables such as organizing, selecting, connecting information, and communicative achievement through planning, monitoring, and evaluating strategies.

• Exploring the Fourth Research Question (Qualitative Results)

The data in the second phase were gathered via audio-recorded, semi-structured interviews that offered insight into the participants’ perceptions and opinions concerning the effectiveness of metacognitive-based assessment instruction, which helped to answer the second research question. Audio recordings of 12 MAIG interviews were made and subsequently transcribed. The data-driven approach was applied in the current study. The following themes were extracted from the participants’ interviews based on thematic analysis:

• Challenges (lack of training, students’ language proficiency, time restriction, and group size) (2) Individual accountability (collaborative learning, self-efficacy, and independent learners), (3) Self-regulation (self-monitoring and independent learners), and (4) Positive interdependence (group work activities, and group feedback).

4.3.1. Challenges

Regardless of whether in the classroom or a large-scale setting, writing assessment is a troublesome activity. In this regard, the interviewees mentioned various facets that could play a considerable role in the writing assessment procedure.

4.3.1.1. Lack of Training and Time Restriction

EFL learners’ training in metacognitive procedures and rubrics requires an accurate and clear-cut description of a set of criteria to assess students’ writing performances. A clear set of criteria and assessment training should be continuously applied to each student’s writing samples to decrease teachers’ biases. Assessors’ subjectivity leads to problems in fairness, accuracy, and reliability of assessments. The proper training of EFL learners will enable them to write effectively, and when they act as teachers, they will be able to help their learners and critically assess their writing tasks. Developing such training would improve the value of teachers’ assessments, and the ultimate advantage could be an autonomous and self-regulated learner. In this regard, one of the interviewees noted:

I believe that the lack of suitable training for writing assessment plays a vital role in decision-making and assessment. Assessment programs should be based on the most recent learning, writing, and assessment research.

Other drawbacks refer to time limitations and group size in each session. Experience and specialized knowledge can be significant in taking care of the issue (Pusparini, Widiati, & Susanti, 2021). As one of the interviewees commented:

The main problem concerns time limitation, and I did not find enough time for discussion and solving the problem. I prefer the number of learners would be more limited in each classroom.

• Language Proficiency and Different Attitudes Toward Writing Assessment

One deficiency of writing assessment is learners’ lack of adequate proficiency level, and their heterogeneity in their beliefs toward writing assessment are the sources of negative attitudes toward writing assessment. However, as Farahian (2017) argues, diversity of perspectives among learners can be beneficial for participants since they tend to promote their collaboration and group work to overcome the obstacles in the learning process. The more knowledgeable learners can motivate novice learners to enhance their confidence and develop positive attitudes toward their decisions in the learning process. In this vein, one of the participants commented:

My main problem is that I didn’t know enough about writing assessments and strategies.

Individual accountability refers to the conviction that each learner is responsible for achieving a common goal and accomplishing a task. Individual accountability is a factor in revealing whether each participant has accomplished the group’s common goal or not and happens wherein learners cooperate in a small group (Ghufron & Ermawati, 2018). It shows that the learners could perform better and achieve the task’s purpose independently after participating in a cooperative class and group activity.

• Collaborative Strategies and Self-efficacy

Individual accountability is considered a primary component of collaborative learning. Collaborative learning happens when learners cooperate toward the same goal of solving problems and completing tasks. Most participants believed that cooperative learning and assessment writing improved their confidence and motivation for more interaction in learning. In addition, individual accountability could enhance the participants’ sense of self-efficacy for more outstanding communication and accomplishment (Ghufron & Ermawati, 2018; Zimmerman, 2013). After taking part in a cooperative course, participants should independently assess and complete the writing tasks. One respondent asserted:

I am always worried about writing assessments. The main problem concerns times when I am not sure of grammatical points or communicative achievement of a text. As a TEFL student, I tend to create a positive attitude toward writing skills. I believe that collaborative assessment can help enhance my self-esteem and confidence within the group.

4.3.2. Self-regulation

Learner-centered Education and Independent Learners

Regarding the importance of increasing metacognitive knowledge awareness such as task, person, and strategic knowledge in EFL classes, most educators believe that person knowledge can play a crucial role in enhancing learning (Schnaubert, 2019). Schnaubert (2019) notes that successful learners have sufficient metacognitive knowledge about themselves as learners, the cognitive task’s nature, and suitable strategies for gaining cognitive purposes. In this respect, Vygotskian theory emphasizes that social interactions with adults or teachers and more knowledgeable peers are the main factors for independent learning. The teacher steadily withdrew her support in the learning process to enable the learners to perform tasks independently. However, participants’ performance of the same tasks under the teacher’s guidance contributed to its internalization. As one interviewee highlighted:

From what I experienced, I can say that learner-centered courses helped me become autonomous. I think traditional teacher-centered practices cannot support learners to gain autonomy.

4.3.3. Positive Interdependence

Positive interdependence is a component of cooperation and collaborative learning. All participants realized that teamwork benefits individuals and groups, and their achievement relies on the participation of all learners (Ghufron & Ermawati, 2018).

Group Activity and Feedback 

Positive interdependence can improve with the teacher’s support and learners’ collaboration and self-control over their learning process. For instance, some learners elaborated on the role of raising metacognitive awareness in preparing them for group activities.

 I learned from my peers’ feedback that thinking together created new ideas. It helped me select the most appropriate method for my improvement and check my errors.

Subsequently, positive interdependence, group work, and peer feedback increased the transparency among the learners. Negotiation with peers or group members would improve their speaking ability and boost their interaction. For example, one of the participants mentioned:

Students’ evaluation improved in group work through peer interaction and the teacher’s scaffolding and feedback.

All in all, the participants welcomed different metacognitive strategies in writing assessments to achieve self-regulation and self-monitoring in their learning process to become independent learners. The learners showed that they could revise the essays in different aspects of the writing, such as surface level, content level, communicative achievement, language level, and the organization of the texts.

5. Discussion

Increasing EFL learners’ awareness of their cognition via developing metacognitive strategies in writing assessment assists them in engaging in learning regulations while assessing. The central part of this study was related to determining how raising metacognitive awareness affected writing assessment. The different statistical analyses led the researchers to conclude that metacognitive strategies positively, directly, and significantly affect participants’ fairness and reasonableness while evaluating an essay.

The rating of an essay is much more than merely scoring it. Mental schemata of decisions, previous knowledge, and knowledge retrieval promote assessment. From the beginning of the writing assessment, decision-making would follow upon knowledge retrieval. Accordingly, Lane, Gobet, and Cheng (2000) consider that knowledge retrieval is a schema in long-term memory. Writing assessment is regarded as a dynamic process that does not depend only on linguistic, rhetorical, and world knowledge but requires metacognitive knowledge and regulations (Khodabakhsh et al., 2018).

The positive impact of metacognitive awareness-raising strategies on EFL learners’ writing assessment indicates that learners should formulate writing schemes to process their decision-making through verbalizing their thoughts. The findings revealed that discussions, think-aloud activities, collaboration with peers and the teacher, questions-answers for problem-solving, planning, monitoring, and revising strategies could effectively boost participants’ metacognitive awareness.

The findings of the MAWQ based on Flavell’s (1979) metacognitive theory showed that the participants’ knowledge of metacognitive strategies before writing assessment is an integral part of the decision-making process regarding what and how they would assess it (planning strategy). During writing assessment tasks, decision-making is essential for implementing aspects of their rhetorical plan. Such knowledge can help them revise (monitoring and evaluating strategies) a composition (Zhao & Liao, 2021). In line with Maftoon, Birjandi, and Farahian (2014) within Flavell’s (1979) framework, this study indicated that the person, task, and strategy knowledge types are beneficial and could contribute to problem-solving and decision-making. The findings substantiate that metacognitive strategy promotes strategy knowledge (Chanski, 2015; Teng et al., 2021). In like manner, Kural’s study (2018) shows that indirect assessment influences learners’ competence.

The results obtained from the administration of the MAWQ showed that the MAIG participants could become aware of the metacognitive knowledge and regulations. They employed such strategies during the assessment process, which resulted in the improvement of self-regulation and self-correction. However, studies focusing on the role of metacognition in writing assessment are scant. In this regard, the findings of this research are consistent with Khodabakhsh et al. (2018), who indicated that dynamic assessment promoted EFL learners’ feedback, interaction, and the solving of the obstacles in writing assessment. Similar to the findings detailed by Oudman et al. (2021), the present study also revealed that metacognitive awareness strategies involved learners in problem-solving. Concerning the study’s findings, metacognition is not only helpful in assessing but can also enhance self-regulation. It helps learners control their emotions and behavior and become self-regulated (Zimmerman, 2002). It is worth considering that the degree of the MAIG learners’ performance and achievement based on the raters’ assessment revealed that metacognitive-based assessment, when compared to cognitive-based instruction, enhances the fairness and accuracy of learners’ decisions in assessing essays. The results showed that the MAIG’s assessment had the highest compatibility with the raters’ mean scores, and they significantly outperformed the post-rehearsed and unrehearsed assessments in the final session.

6. Conclusion and Implications  

The present study showed that writing assessment through raising metacognitive awareness is a dynamic process in which learners can monitor, evaluate and identify the challenges in the writing assessment process. Critical thinking and the meaningful learning assessment process improve learners’ self-regulation, achievement, and writing assessment decisions. Second, the two facets of metacognitive knowledge and regulations are teachable. In addition, they may also be helpful strategies in essay assessment and achieving cognitive goals.

EFL teachers, learners, materials developers, and syllabus designers may be the beneficiaries of the current study. They can gain insights into how far raising metacognitive awareness can improve English writing assessment. The findings would help EFL learners learn about their learning, improve learning strategies, and benefit from them in new situations. Meanwhile, this research can help EFL teachers create learning situations to provide proper instruction, practice, and constructive feedback to train independent, self-regulated learners. Moreover, this research can be substantial to material developers and syllabus designers and stimulate them to design materials that help EFL learners become self-directed, self-regulated, and self-evaluative. The study can also draw their attention to design relevant activities. Moreover, after the COVID-19 pandemic, most educational systems can easily shift from face-to-face to modular or online learning via practicing metacognitive strategies since different learning outcomes, such as directed attention, self-management, self-monitoring, and cooperation, are empowered.

Further research can investigate the effects of metacognitive knowledge and regulations on other skills’ assessments. This study only probed EFL learners’ awareness and reactions using metacognitive knowledge and strategies through essay measurements and a questionnaire. The findings can be further researched through performing interviews and observations with teachers. Future research studies can also focus on studying the effect of mobile-assisted language learning on EFL learners’ cognitive and metacognitive strategy use. In addition, further studies can be performed for learners with brain injury or memory impairment to support them in becoming successful in education through utilizing metacognitive strategies and enhancing self-regulation.

Eventually, it is noteworthy to mention that one of the most critical limitations of this research was the outbreak of the COVID-19 pandemic, which plagued educational systems worldwide, leading to the near-total closing. Therefore, the researchers had to use online instead of in-person classes. Besides, considering individual differences such as age, motivation, and social background could allow the researchers to interpret the findings more profoundly. The final concern was that injecting, to some degree, a set of pre-defined cognitive and metacognitive writing strategies would deprive students of their creativity.

7. Acknowledgment

The authors acknowledge the editors of the Iranian Journal of English for Academic Purposes for giving us the opportunity to submit this manuscript. In addition, we would like to offer our sincere gratitude to the Head of the English Language Department of Islamic Azad University (IAU), South Tehran Branch, and all the study participants for their tremendous cooperation and support.

8. Declaration of Conflicting Interests

The researchers declare no conflict of interest for the present paper.

9. Funding Details

The researchers received no funding to declare.

References

Beiki, M., Raissi, R., & Gharagozloo N. (2020). The differences between Iranian EFL teachers’ perceptions and their instructional practices regarding cooperative learning, Cogent Arts & Humanities, 7(1), 1-30. https://doi.org/10.1080/23311983.2020.1847420

Capel, A., & Sharp, W. (2014). Objective first students’ book with answers with CD-ROM. Cambridge, MA: Cambridge University Press.

Chanski, S. (2015). Assessing writing through metacognitive and reflective practice. Language Arts Journal of Michigan, 31(1), 53-56. https://doi.org/10.9707/2168-149X.2097

Creswell, J. W. (2021). A concise introduction to mixed methods research. SAGE publications.

Dinsmore, D. L., & Alexander, P. A. (2012). A critical discussion of deep and surface processing: What it means, how it is measured, the role of context, and model specification. Educational psychology review, 24(4), 499-567. http://doi.org/10.1007/s10648-012-9198-7

Farahian, M. (2017). Developing and validating a metacognitive writing questionnaire for EFL learners. Issues in Educational Research, 27(4), 736-750. https://www.researchgate.net/publication/320830404

Filed, A. (2018). Discovering Statistics Using IBM SPSS, Statistics for Statistics. (5th ed.). London: SAGE Publications.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906-910. https://doi.org/10.1037/0003-066X.34.10.906

Ghufron, M. A., & Ermawati, S. (2018). The strengths and weaknesses of cooperative learning and problem-based learning in EFL writing class: Teachers’ and students’ perspectives. International Journal of Instruction, 11(4), 657-672. http://dx.doi.org/10.12973/iji.2018.11441a

Khodabakhsh, S., Abbasian, G. R., & Rashtchi, M. (2018). Incorporation of dynamic assessment models into developing language awareness and metacognitive strategy use in writing classes. Journal of Modern Research in English Language Studies, 5(4), 55-79. https://doi.org/10.30479/JMRELS.2019.10826.1353

Kural, F. (2018). Does indirect writing assessment have any relevance to direct writing assessment? Focus on validity and reliability. Journal of Language and Linguistic Studies, 14(4), 342-351. https://doi.org/10.1016/j.lmot.2017.02.007

Lane, P., Gobet, F., & Cheng, P. (2000). Learning-based constraints on schemata, In Proceedings of the Twenty-second Annual Meeting of the Cognitive Science Society., 776-781. https://doi.org/10.3389/fpsyg.2015.01785

Lee, I., & Mak, P. (2018). Metacognition and metacognitive instruction in second language writing classrooms. TESOL Quarterly, 52(4), 1085-1097. https://doi.org/10.1002/tesq.436

Maftoon, P., Birjandi, P., & Farahian, M. (2014). Investigating Iranian EFL learners’ writing metacognitive awareness. International Journal of Research Studies in Education, 3(5), 37-52. https://doi.org/10.5861/ijrse.2014.896

McMillan, J. H., & Hearn, J. (2008). Student self-assessment: The key to stronger student motivation and higher achievement. Educational Horizons, 87(1), 40-49.

Nemati, M., Ghafoori, M., Birjandi, P., & Izadpanah, S. (2021). Self-assessment, peer assessment, teacher assessment and their comparative effect on EFL learners’ second language writing strategy development. Two Quarterly Journal of English Language Teaching and Learning University of Tabriz, 13(28), 201-216. https://doi.org/10.22034/ELT.2021.48543.2456.

Oudman, S., van de Pol, J., & van Gog, T. (2021). Effects of self-scoring their math problem solutions on primary school students’ monitoring and regulation. Metacognition and Learning, 17, 213-239. https://doi.org/10.1007/s11409-021-09281-9

Pusparini, R., Widiati, U., & Susanti, A. (2021). Pre-service teachers’ beliefs about English language teaching and learning in EFL classroom: A review of the literature. Journal of English Educators Society, 6(1), 147-154. https://doi.org/10.21070/jees.v6i1.1212

Rashtchi, M., & Khoshnevisan, B. (2020). Lessons from critical thinking: how to promote thinking skills in EFL writing classes. European Journal of Foreign Language Teaching, 5(1), 34-47. https://doi.org/10.46827/ejfl.v5i1.3153

Santos, L., & Semana, S. (2015). Developing mathematics written communication through expository writing supported by assessment strategies. Educational studies in mathematics, 88(1), 65-87. http://doi.org/10.1007/s10649-014-9557-z

Schnaubert, L. (2019). Providing cognitive and metacognitive awareness information to support regulation in individual and collaborative learning settings [Doctoral dissertation], Universität Duisburg-Essen. https://duepublico2.uni-due.de

Teng, M. F., Qin, C., & Wang, C. (2021). Validating of metacognitive academic writing strategies and the predictive effects on academic writing performance in a foreign language context. Metacognition and Learning, 3(189), 1-24. https://doi.org/10.1007/s11409-021-09278-4

Vygotsky, L., & Cole, M. (1978). Mind in society: Development of higher psychological processes. Cambridge, MA: Harvard University Press.

Xiao Y. (2007). Applying metacognition in EFL writing instruction in China. Reflections on English Language Teaching, 6(1), 19-33. https://www.researchgate.net/publication

Zhao, C. G., & Liao, L. (2021). Metacognitive strategy use in L2 writing assessment. System, 98(3), 102472. https://doi.org/10.1016/j.system.2021.102472

Zimmerman, B. J. (2013). From cognitive modeling to self-regulation: A social cognitive career path. Educational Psychologist, 48(3), 135-147. https://doi.org/10.1080/00461520.2013.794676

Appendix A: Writing Rubric for B2 (WR)

Preliminary Writing Scale of Cambridge Assessment English regulated with explicit reference to the Common European Framework of Reference for Languages (CEFR) (2019)

Appendix B: Interview Questions (Writing Assessment)

Knowledge of cognition

1. What is your general attitude toward writing assessments?
2. What kind of problems do you often encounter while assessing a piece of writing in English? What is the main one? How do you deal with your problems?
3. Are you aware of the strategies you use while writing assessments? Do you have specific reasons for using them?
4. If you use specific strategies, when and why you use them. How do you apply the strategies to your writing assessment?

Regulation of cognition

1. What do you do first before you begin writing assessments in English? Do you have any plan in your mind before starting to assess? Explain.
2. While assessment, what do you do when you have a problem and get stuck? Do you consider several alternatives to the problem?
3. Do you often stop while assessing a writing task and ask yourself how well you are doing? What do you do then?
4. Do you often reevaluate your assessment?
5. What do good writing assessors do?
6. How often do you utilize scoring rubrics, and has your opinion changed any since you have applied them?

How has the application of scoring rubrics impacted your performance?

[1]Ph.D. Student of TEFL, mojgan.kouhpayeh@gmail.com, TEFL Department, Faculty of Foreign Languages, Islamic Azad University, South Tehran Branch, Tehran, Iran.

[2] Associate Professor, mojgan.rashtchi@gmail.com, TEFL Department, Faculty of Foreign Languages, Islamic Azad University, North Tehran Branch, Tehran, Iran.

[3] Professor, rostamiabu@uk.ac.ir, TEFL Department, Faculty of Literature and Humanities, Shahid Bahonar University of Kerman, Kerman, Iran.

[4] Assistant Professor, mowlaiebahram@gmail.com, TEFL Department, Faculty of Persian Literature and Foreign Languages, Islamic Azad University, South Tehran Branch, Tehran, Iran.