1. Introduction

Automated Writing Evaluation refers to the computer tool that is used to give formative feedback in various contexts [1]. By scoring the writing, this kind of technological program can analyze the grammar, structure, and other aspects and produces detailed evaluation and feedback based on the text. With its simple operation step and instant corrective feedback, AWE tools, such as Criterion, MY Access!, and Grammarly are applied in language-related aspects and used frequently in terms of scoring papers as the human rating is considered to be time-consuming and inconvenient.

The company Vantage Learning made My Access! — AWE software, which is a website that provides holistic and component learning. Through Artificial intelligence, My Access! offers instructional feedback for people and it also provides hundreds of prompts for users of different levels. Criterion serves as another instructional tool, which is based on Natural Language Processing as the evaluation mechanism. With a wide range of individualized feedback, this system contains four categories: grammar, usage, style, and mechanics [2]. Meanwhile, it also includes many subcategories, such as “confused words”, and “repetition of words”. As a free online writing assistant, Grammarly is capable of providing text scores as well as classifying all suggestions into different standards: correctness, engagement, clarity, and delivery. After processing the document, it will not only underline these issues in colors but also show users how to correctly write the sentence.

Automatic scoring systems have been developed since 1960 to serve as a grader for students’ writing and help teachers to correct papers with instant feedback and detailed instructions [3]. Using various writing technologies in the classroom can reduce the excessive workload of teachers and arouse learners’ interest in writing because of the creative teaching method. The adoption of AWE in the classroom can also cause negative effects. Without real interactions, students cannot tell their ideas or arrangement of the paper or discuss revisions, thus failing to improve the overall writing process [4]. In addition, students’ autonomy learning in the class will be hampered as the AWE tool modify the work with abundant resources and editing skills, in the sense that learners will accept the feedback directly without further thinking [5].

AWE has a close relationship with education as it has developed since 1960, with the aim to save teachers’ time when grading written work and provide feedback on the student’s essays [3]. Indeed, AWE is believed to have a positive impact on learners’ writing ability because it allows teachers to spend more time explaining writing prompts and giving writing instructions, thus boosting teaching efficiency [6].

This review paper mainly focuses on AWE tools and their impact on learners’ English writing ability. The significance of this paper is to help people deepen their understanding of AWE tools, fully analyzing the effects on their test performance, writing revision, and writing self-efficacy, thus helping learners to use AWE technology in a more effective way. In addition, with the assistance of technological tools, learners can make progress in holistic English writing and revision.

2. Effects of Automated Writing Evaluation

2.1. Test Performance

Test performance tends to be the direct indicator of students’ writing ability. Students are required to participate in various tests, to help researchers find out the effects resulting from AWE tools. Wilson and Roscoe made investigations from The Smarter Balanced ELA test and finds out that Project Essay Grade positively affects test performance [7]. From the PEG score, participants would modify mistakes and restructure the writing according to the evaluation, thus achieveing better grades on the test. Link, Mehrzad, and Rahimi compared the AWE Team and the Teacher Team [8]. After the first-test, and last-test, the AWE group improved their writing accuracy, and the Teacher group also showed progress in complexity, accuracy, and fluency. However, when it comes to long-term impact, the AWE group performed better as these participants get higher grades in a three-month latter delayed post-test. Researchers believe that the nature of AWE plays an important role because students are more likely to internalize writing knowledge from AWE’s constant feedback. Teachers are familiar with students’ original writing abilities and are good at giving detailed instructions. These participants can get more comprehensible feedback than they get the AWE tool, in the sense that they are more likely to achieve better grades on tests. However, the instructions from AWE may be clearer and more acceptable as students can see the revised sentences directly, which can encourage them to internalize the key points that they have got and reduce mistakes in the tests in the long term.

Lee analyzed two students’ writing test scores and compared the data before and after using an automated writing evaluation tool named “Criterion” in South Korea [9]. By comparing participants’ TOEIC writing scores and Criterion scores, the results of the experiment found clear support for the improvement of students’ test scores with the help of AWE tools. Both participants received higher marks in official tests later on. In another research, Parra and Calero also argue that AWE tools like Grammark and Grammarly can greatly improve learners’ writing ability [1]. They collected the test results from participants taking the same examination at different times, intending to indicate their English writing performance. The results lead to a similar conclusion where applying AWE tools can bring a positive influence on learners’ writing skills. AWE tools usually provide advice on aspects such as vocabulary, grammar, logic development, organization, mechanics, punctuation, techniques, and styles. Learners got immediate feedback from AWE tools, which was helpful to seek replaceable choices for their errors and correct them. By receiving an instant response, students are encouraged to acknowledge errors in their writing and pay more awareness to grammar mistakes, editing issues, or other problems without wasting time on waiting for feedback. With the assistance of feedback provided by AWE tools, incorrect parts could be corrected in time. This is also an effective way to improve the student’s test performance.

However, there are reports about AWE tools that might go wrong and offer incorrect suggestions for students. At the same time, various AWE tools can throw out diverse proposals. There are possibilities for the receivers to follow the advice and make mistakes. In contrast, this makes it possible for self-directed learning. In the interview with one of Lee’s participants, Song claimed that she learned a lot from the false feedback because to affirm that the AWE system did make mistakes, she had to search for information such as websites, dictionaries, and other materials [10].

Overall, some incorrect feedback from AWE cannot be ignored, but AWE technologies can make contributions to classroom teaching and improve writing ability and accuracy in the long term, thus encouraging students to perform better on tests.

2.2. Student Revision

AWE plays an important role in many aspects, including student revision. Students tend to correct errors in their papers according to instructions from AWE tools, in order to improve writing accuracy and enhance their writing ability. Li, Link, and Hegelheimer studied 70 students from two ESL writing courses [2]. According to the findings, AWE can encourage students to deepen their understanding of errors. Researchers point out that students make fewer mistakes from several drafts and also find out that when students are allowed to predict their scores before submitting, they could effectively use feedback from Criterion. By setting expectations, students can deepen their understanding of their current writing ability, which helps to facilitate writing revision and self-reflection. Moreover, it also points out that the results of student revision may be related to students’ writing complexity [2]. It is an important factor as the complex structures and grammar that students have employed in the process of writing may lead to more errors considering their writing ability. These students may receive more comprehensive AWE feedback and progress faster in revision compared with participants who apply less complex writing. Link, Mehrzad, and Rahimi hold the view that Criterion has a positive impact on student revision practices that students not only correct superficial mistakes but also change high-level language features [9]. According to this point of view, the systematic feedback and students’ proficiency in using the software may serve as motivations for them to analyze the logic and organization of their papers. Furthermore, the development of technology provides AI with an opportunity to grow, to progress. Another research done by Parra and Calero indicates that AWE tools play an important role in assisting the teachers’ feedback [1]. AWE tools provide autonomy as well as encouragement for students to learn how to write and how to write better on their own. The descriptive statistics show participants’ high agreement on the positive impact when using AWE tools. Participants claimed that the AWE tools aid them to enhance their writing accuracy with help of those viable alternatives. It can be predicted that AWE can provide autonomy well as encouragement for students to learn how to write and how to write better on their own. Through AWE tools, students receive formal suggestions which are related to their writing errors. At the same time, feedback doesn’t include personal emotions. Students’ autonomy is prevented from being influenced by feelings. Seeing themselves doing better, learners gain more confidence and willingness to continue their study. This kind of personalized feedback not only expedites students’ access to advice but also brings privacy to students [1]. Chen and Cheng point out that AWE tools aim to cultivate students’ autonomy via misdiagnosis of learners’ input, collecting learners’ personal feedback, and providing specialized access to correcting different categories of errors [5].

Geng and Razali argue that the Pigai program carries out the related writing knowledge [10]. Through individualized real-time feedback, instead of letting this related knowledge stay in short-term memory, AWE tools transform them into long-term memory. Students revise their articles after accepting the immediate diagnostic and formative feedback from AWE tools. In this case, AWE tools can enhance the student’s writing skills by offering repeated writing practices. What’s more, using artificial intelligence creates a systematic review of students’ revision. Students can analyze their errors and find out certain aspects where they often made mistakes. Instead of editing all aspects of written works, it’s more efficient to concentrate on several particular sides of writing such as grammar, punctuation, and vocabulary.

However, Link, Mehrzad, and Rahimi also finds that more feedback from AWE is to remove, which belongs to conservative instruction [8]. It can be predicted that too many words or sentence reduction is of little use to the improvement of revision and students may avoid facing writing problems directly, thus the AWE system has not been well-constructed yet.

It can be concluded that AWE tools have a positive impact on student revision practices. They provide autonomy as well as encouragement for students to learn how to write better on their own. Meanwhile, students should be critical of the feedback they have received.

2.3. Writing Self-efficacy

Self-efficacy means the level of confidence individuals have in coping with matters. A high level of writing motivation and self-efficacy is beneficial to learners’ writing in all aspects. They are more likely to make better preparations for the papers they are going to write. Meanwhile, when students come across unfamiliar topics, they would not feel anxious easily and can face the challenge directly, making full use of all the writing materials with confidence [11]. Besides, Lv, Ren, and Xie reveals that writing self-efficacy is influenced by variable factors, including, writing test, AWE feedback, and other points [12]. When students come across difficult topics, their writing efficacy may decrease as they feel hard when writing an outline or the main content. Meanwhile, positive feedback from AWE tools can boost students’ confidence as they can modify papers according to detailed instructions, while incorrect online feedback may serve as a barrier to people’s advancement. According to Wilson and Roscoe, the “composing condition”, whether students use Google Docs or PEG influences students’ posttest writing efficacy in a small and direct way [7]. Posttest self-efficacy can be predicted from pretest self-efficacy and students in the PEG group have better writing self-efficacy. Students’ self-efficacy is persistent and will not change dramatically when using the AWE tool. In addition, under the guidance of AWE, there would be fewer grammatical or logical mistakes in writing, thus students will have more confidence in writing tasks compared with the Google Docs group.

Bruning, Dempsey, Kaufmann, McKim, and Zumbrunn used Self-Efficacy for Writing Scale (SEWS) to probe into students’ writing self-efficacy [13]. It consists of 16 items, which can be concluded into 3 categories: convention, ideation, and self-regulation, and students rated the self-efficacy from 0 to 100. Results show that participants are more confident in terms of convention and have the least confidence in the self-regulatory aspect [7]. It shows that learners feel easier when dealing with writing accuracy and complexity than the feelings evoked by writing. As a result, it can be predicted that when AWE have successfully addressed language-level problems for students, they may become more positive when struggling with writing anxiety or other feelings.

Research about how Pigai.org influence and improve English learners’ writing self-efficacy was designed by Li and Yan [14]. In their research, 39 sophomores who majored in English attended and completed questionnaires related to English writing self-efficacy. By analyzing attendants’ self-efficacy average scores in the pre-questionnaire and post-questionnaire, Li and Yan concluded that participants’ writing self-efficacy had been improved via the use of the AWE tool Pigai.org.

In the era of high-tech, Internet-based technologies make our lives easier. In terms of writing, online writing and applying AWE tools bring convenience to learners. At the same time, because of the improvement in self-efficacy, students show more interest in the form of online writing than the conventional writing model. AWE tools play a supporting role in learners’ writing, which makes students’ writing more efficient. Seeing themselves doing better, students have more enthusiasm for writing. AWE tools boost learners’ confidence in writing. Since AWE tools are based on electronic products, proficiency of software applications is a vital factor in enhancing learners’ interest in online writing. In term of other aspects, Li claimed that computer self-efficacy shows positive influences on perceiving ease of use [15]. Learners experience more easiness as they become more confident in their computer skills. Li et al. argue that the EFL learners’ sense of easiness is greatly affected by computer self-efficacy [16]. Students would pay more attention to their writing and therefore their writing skills can be significantly improved. What’s more, learners experience more happiness in writing, which can also encourage them to keep on writing instead of giving up easily. AWE tools give learners confidence and foster writing self-efficacy via providing immediate scores and instant feedback, offering privacy and autonomy for students, and using statistic data to analyze the students’ improvement. Besides, the advancement of writing self-efficacy is proven to be beneficial to learners’ writing ability [17].

3. Conclusion

The authors mainly generalize the positive impacts that AWE tools have on test performance, revision, and writing self-efficacy. Among the research above, students who are available to AWE tools perform better in tests and get higher marks. By receiving instant customized feedback which helps and develop their writing abilities through diverse approaches like grammar, writing complexity, and techniques, English learners can revise their writing more effectively. Applying AWE tools to students’ revision can systematically review their errors, helping students to modify the inaccuracy part. Though students’ writing self-efficacy will not be dramatically influenced by the use of AWE tools, students are more confident about themselves and encouraged to keep on writing under the lead of AWE tools. Proficiency in using computers can also enhance computer self-efficacy, which is beneficial to improve comprehensive writing ability. Meanwhile, students who are less proficient in English improved more significantly than those at a higher level. Students who are interviewed agree or highly agree that their confidence in writing has been increased after taking the positive feedback given by AWE tools. Through this paper, students can deepen their understanding of AWE tools, helping themselves to use AWE technology in a more effective way. In addition, it provides theoretical support to apply AWE in the traditional classroom. However, current studies still focus on the results of students’ writing competence, writing performance, and the factors which might influence English learners’ writing study after using AWE tools. Given the conservative suggestions that AWE provides, more researches on the its negative effects and how students can discriminate provided feedback should be done.