نوع المستند : المقالة الأصلية
المؤلف
Assistant Professor of Curriculum and EFL Instruction Hurghada Faculty of Education South Valley University
المستخلص
الكلمات الرئيسية
الموضوعات الرئيسية
كلية التربية
إدارة: البحوث والنشر العلمي ( المجلة العلمية)
=======
A Program Based on Chat Generative Pre-trained Text Transformer (ChatGPT) for Enhancing EFL Majors' Descriptive Paragraph Writing Skills and Their English Grammar Use
By
Dr. Haggag Mohamed Haggag
Assistant Professor of Curriculum and EFL Instruction
Hurghada Faculty of Education
South Valley University
doctorhaggag@hu-edu.svu.edu.eg
} المجلد التاسع والثلاثون– العدد السادس – يونيو 2023م {
http://www.aun.edu.eg/faculty_education/arabic
المستخلص:
هدفت الدراسة إلي التعرف علي فعالية استخدام برنامج قائم علي المحوِّل التوليدي مسبق التدريب للمحادثة لتحسين مهارات كتابة الفقرة الوصفية واستخدام قواعد اللغة الإنجليزية لدي طلاب الفرقة الثالثة بشعبة اللغة الإنجليزية. تمثلت عينة الدراسة في عينة قوامها 30 طالبا بالفرقة الثالثة بكلية التربية بالغردقة حيث تم اختيارهم عشوائيًا. واتبعت الدراسة التصميم شبه التجريبي ذا المجموعة الواحدة بقياسيه القبلي والبعدي. قام الباحث بتصميم أدوات ومواد الدراسة والتي اشتملت علي: استبانة مهارات كتابة الفقرة الوصفية ، اختبار مهارات كتابة الفقرة الوصفية، بطاقة اختبار كتابة الفقرة الوصفية (وصف الأماكن – الأشياء، الأحداث والأشخاص)، بينما تكون اختبار قواعد اللغة (علامات الترقين – الأزمنة – بنية الجمل). توصلت نتائج الدراسة إلي وجود فروق ذات دلالة احصائية بين متوسطات درجات العينة في القياسين القبلي والبعدي لمهارات كتابة الفقرة الوصفية واستخام قواعد اللغة الإنجليزية لصالح الاختبارات البعدية. وتوصي الدراسة باستخدام المحول التوليدي مسبق التدريب في تنمية جوانب لغوية أخري مثل الكتابة الناقدة، التلخيص، وإعادة الصياغة. كما توصيالدراسةبإجراء بحوث حول استخدام المحوِّل التوليدي مسبق التدريب في التقويم المبني علي الأداء لدي المعلمين قبل وأثناء الخدمة.
الكلمات المفتاحية
المحوِّل التوليدي - مسبق التدريب - الكتابة الوصفية – قواعد اللغة
Abstract
This study aimed to identify the effectiveness of using a ChatGPT based program for developing descriptive writing skills and English grammar use of 3rd year EFL Majors. Participants of the study (30) were randomly selected from English department at Hurghada Faculty of Education. The study followed the one-group quasi-experimental design with its pre and post testing procedures. Instruments and materials of the study were designed by the researcher which included: a descriptive writing skills questionnaire, a descriptive writing test, a descriptive writing rubric, an English Grammar use questionnaire, an English grammar use test and a Chat-GPT oriented program. The descriptive writing skills test included place, things, events and character description, while the grammar use test included punctuation, tenses, and sentences structure. Results of the two tests showed significant means differences between the participants in the pre and post testing procedures favoring the post testing. This means that there were significant mean differences between the participants in both descriptive paragraph writing skills and English grammar use due to the ChatGPT driven program of the study favoring the post application. The study recommended using ChatGPT tool for other language areas development such as critical reading, summarizing and paraphrasing. It also recommended running further research on using ChatGPT for performance based assessment of pre and in-service teachers.
Key words: ChatGPT- Descriptive Writing – English Grammar.
Introduction
Artificial Intelligence AI is widely implemented in different scientific fields and disciplines including foreign language education. This fast growing use increases the concerns about the way education should adopt or adapt this important discipline. The vast use of AI-powered educational software emerged in different teaching aspects such as writing and grammar. This use changed the learning goals, practices and assessments, which consequently led to different questions about what apps to use and how to use them correctly and effectively in different language areas and skills.
Artificial Intelligence AI (Zhai, 2022) refers to "the ability of computers and other machines to perform tasks that would normally require human intelligence, such as learning, decision-making, and problem-solving" (p.2). This growing discipline has been used differently in Foreign Language Learning (FLL) though its programs, websites, tools and applications. This use can be obvious in the development of the language skills, grammar, and vocabulary (see Azaria, 2022; Kengam, 2020 and Saleh, 2019). In November 2022, OpenAI released ChatGPT as its latest AI-based chatbot.
Chat Generative Pretrained Transformer (Chat GPT) is a state-of-the-art language processing Artificial Intelligence (AI) model developed by OpenAI Company. A chatbot is "a software program that uses user input to simulate human-like interactions. The ChatGPT Chatbot was created by San Francisco-based OpenAI. On November 30, 2022; the Chatbot was made available for free public testing" (Chinonso, 2023, p.10). ChatGPT has revolutionary effect on different scientific disciplines including language education.
AI-based text generators including ChatGPT are useful for foreign language learning in particular. There are different studies that confirm the positive impacts of AI based text generators on FLL (see Haristiani & Danuwijaya, 2019; Hung, Huang, Hsu & Shen, 2016; Wang & Petrina, 2013). For instance, Kim, Young & Cha (2021) stressed that these tools provide participants with opportunities to practice their language skills. Chatbots can provide foreign language learners with interactive chances to develop their grammar use. These tools have also positive impacts on the development of grammar use through the interactive feedback that they provide. Not only can they develop vocabulary and grammar areas, such tools can also develop learners' communicative competence as well. In their study, Kim, et al. (2021) investigated the effect of AI-based text generation tools on EFL students' writing performance. The study used an AI based tool that resulted developments in students' writing assignments and written performance. Text generation tools can enhance students' writing and expressive abilities. The study confirmed the effectiveness of the tools in different writing performance areas such as idea generation, editing and revising.
Using AI based tools and computer technologies have been widely used in EFL literature with the aim of enhancing students' grammar (see Hakmy & Ahmed, 2019; Abo Laban & Firwana, 2017; El Remaly, 2017; Samir, 2015). For instance, Nutta (2001) compared between the teaching of English grammar using computer based technology and traditional class teaching practices. Results showed the positive impact of technology in grammar instruction compared to traditional grammar teaching techniques. Students' performance was higher in CALL driven open ended questions rather than teacher-driven questions. These early results confirmed that using technology for grammar development is a key aspect in TEFL.
Grammar use is a crucial component in any EFL class due to its role in effective communication. In this view, El Kenany (2021) stated that "Grammar plays an important role in language teaching. Grammar helps students speak English with organized and correct structures and thus develop the communicative skills of the language; the problem with grammar is how to teach and present grammar to learners" (p.61). This shows that using grammar in EFL settings should be authentically contextualized with the help of recent teaching approaches and effective technologies such as Artificial Intelligence tools.
Artificial Intelligence coined the recent teaching of grammar and its use with different advantages faced with challenges. Gawate (2019) asserted that using AI generated tools in teaching and learning language including grammar is due to different advantages. The first advantage is the contextualized needs that meet the demands and aspirations of the students. The second is the qualitative content which covers the different areas of the language. The third is the humanized assistance and instant feedback. The fifth advantage is the change in teachers' roles to facilitators and guides to support the learner through these tools. The seventh is the personalization of learning; while the eighth is providing different platforms that meet the different paces of learners. As to the challenges of implementing AI tools (Liu, 2023), they include: the challenge of creating effective interpersonal communication with the student, the challenge of lacking the humanized teaching approach, and the technical limitations such as security risks. The benefits gained from AI generated tools and instruments are more than the challenges they encounter.
Based on the above merits of AI and ChatGPT tools in FLL, this study aimed at identifying the effect of a program based on ChatGPT text generators to develop EFL majors' descriptive paragraph writing skills and English grammar use.
Problem of the Study and its Context
Context of the problem stemmed from three main sources: the first is the researchers' experience in teaching EFL majors at Hurghada Faculty of Education. Students face challenges in writing essays and paragraphs during their Essay and Linguistic Drills courses, which they study during the four years of study. These challenges include correct paragraph writing including descriptive paragraphs which they write around persons, events and places. Other challenges include coherence, cohesion and editing difficulties, which consequently affect the structure and quality of their written paragraphs.
Secondly, through piloting the participants using writing quiz for EFL majors (60 students) at Hurghada Faculty of Education, South Valley University. The quiz included writing three different descriptive paragraphs describing a place, character, thing, and an event. Results showed that (90 %) of the students faced paragraph writing difficulties such as cohesion and coherence as well as grammar and vocabulary issues. EFL majors faced difficulties in writing paragraphs using process writing approach by following pre-writing – during and post writing procedure. Another result showed that (86.6%) of the participants faced challenges in the appropriate use of adjectives and adverbs in describing places, characters, things and events. Grammar use was also a key difficulty in participants' writing; their paragraphs included errors in correct tense use, linking word errors and other sentence structure related areas.
Thirdly, reviewing literature (e.g. Chinonso, 2023; Hong, 2023; Shijaku1 and Canhasi, 2023; Zhai, 2022; Saleh, 2019; Frutos, 2014) showed that there has been various studies that examined the use of AI based tools in developing EFL skills but not descriptive writing areas in particular. There has been a research gap in integrating recent text generators –which are relatively new tools- in developing EFL writing and grammar use.
In the light of the above context, problem of the study can be stated in the challenges that some EFL majors face in their descriptive paragraph writing and English grammar use. There is a need to design a program based on AI tools and text generators to develop EFL majors' descriptive paragraph writing skills their use of English grammar.
Objectives of the Study
The study aimed to the following:
Significance of the Study
This study is thought to be significant for the following:
Questions of the study
Hypotheses of the study
Delimitations of the Study
The study was delimited to the following delimits:
Definitions of the Study
ChatGPT was defined by Demir & Zaremba (2023) to mean "a large language model developed by OpenAI, which is based on the transformer architecture and fine-tuned on a massive amount of data. The transformer architecture is a deep neural network that uses self-attention mechanisms to process input sequences, which allows it to handle variable-length input and output sequences, and to generate human-like text" (p.1). It is one of the resent Artificial Intelligence tools that are used in Foreign Language FL education.
Wyrick (1978) early defined descriptive writing to mean "a form of writing that creates a word picture of persons, places, objects, and emotions using a careful selection of details to make impression on the reader' (p. 42). Descriptive writing includes describing people, things and events using senses and factual-oriented vocabulary rather than opinion and judgment-oriented wording.
English grammar use refers to the practical application of rules that guide the formation and use of English language. Grammar usage (Stephens (2023) refers to "how words are used in sentences, which applies to both written and spoken language; this includes word order, subject-verb agreement, tenses, and the like" (p.2). English grammar use refers procedurally to the different drills, activities, and language forms practiced by the students using technology-enhanced environments.
Method
This section deals with the design of the study, the participants and their selection, instruments and materials of the study and their validation, and the procedures followed in the study in the light of the quasi-experimental design of the study. In addition, it describes the program of the study and its use.
Design
This study follows the one-group quasi-experimental design with its pre and post testing procedure due to the nature of the study. It was applied over one group of EFL students and they were apt to pretesting in areas of descriptive paragraph writing and grammar use using the designed test. Later, they were trained using a ChatGPT based program, and then they were tested using the same tests of descriptive paragraph writing and English grammar use.
Participants
Participates of the study were 30 third year EFL majors at Hurghada Faculty of Education. They were adjusted according to specialization (English) and grade (3rd year Majors). They were grouped in one experimental group; they were instructed using the program based on ChatGPT based program.
Instruments and Materials of the Study
This study utilized the following five instruments and materials which were designed by the researcher and were validated in terms of content validity and reliability (refer to appendix) as follows:
The questionnaire aimed to identify the descriptive writing skills that are needed for the participants. It was validated by a jury of TEFL experts as they added their comments, suggestions and modifications to both the items and layout that serve the aim of the instrument. Reliability of the questionnaire was calculated using Cronbach Alpha statistic (r) as the value was (0.91), which is a high reliability coefficient. The final version of the questionnaire included (21) items that describe the descriptive writing skills that are needed for the participants (see appendix).
The questionnaire aimed to identify the grammatical areas that are essential for 3rd year EFL majors in the light of the grammar course description. After running content validation by a jury of TEFL experts, and running the modifications based on their comments, suggestions and adaptations, the following three areas were included: a. punctuation use, b. tenses use and c. sentences use. Reliability of the questionnaire was calculated using Cronbach Alpha statistic; (r) value that was (0.88), which is a high reliability coefficient. After validation, the final form of the questionnaire included 9 punctuation items in the first part, while the second part (tenses use) included 8 tense use items, and the last part (sentences use) included 9 items (see appendix).
This descriptive writing test aimed to assess 3rd year EFL majors' descriptive writing skills based on ChatGPT text generator. The test consisted of four questions about places, events, characters, and things. Participants were asked to respond to the test using: a. ChatGPT prompts, b. first draft, and c. edited draft. Test time was calculated by dividing the total time taken by sample examiners over their number, which was (60) minutes. Content validity of the test was calculated by a jury of TEFL experts who added their comments, suggestions and adaptations to the test. Reliability of the test was calculated by using Cronbach Alpha statistic (r) value was (0.880), which is a high reliability coefficient.
It was used to correct participants' written texts. The rubric was validated by a jury of TEFL experts and it included the following domains: a. ideas, b. organization, c. introduction, d. word choice, e. sensory details, f. figurative language, g. sentence fluency, h. conventions, and i. voice. Using a 4-likert scale, the raters were asked to assess the responses of the participants in the different areas of the rubric using the set performance indicators.
The test aimed to assess participants' use of grammar. The test was validated by a jury of TEFL experts and the reliability of the test was (0.970), which indicates the high reliability of the test. It included 3 main areas (punctuation- tense use and sentence structure). Test time was calculated by dividing the total time taken by sample examiners over their number, which was (60) minutes. The test included 6 punctuation mark questions, 15 tense use items and 10 sentence use items (see appendix).
The program of the study was used to develop participants' descriptive paragraph writing (places, characters, things, and events), and English grammar use. The program was validated by a jury of TEFL experts who reported its validity after running the modifications and suggestions regarding the objectives, tasks and content areas included in the program. Some recommendations included integrating the different areas of the program in the tasks and not to separate tasks, others included grammar and spelling errors. The final version of the program included ChatGPT technical theme, ChatGPT writing, ChatGPT for grammar, and ChatGPT for language use. The program is based on Chat Generative Pre-trained Text Transformer (ChatGPT) tool. This AI-based tool helps the students to generate texts using certain commands from millions of stored databases. Participants are asked to use this tool in practicing descriptive paragraph writing regarding describing people, things and events using in the light of the outputs and their editing techniques. The program also aims to develop their English grammar use through the AI-based outputs regarding punctuation, tense use and sentence structure practices. During the program, participants are asked to generate texts using ChatGPT commands and then editing the text followed by writing final version of the edited text.
Theoretical Framework of the Study
The following review aims to integrate between pertinent literature and theoretical findings that are related to three main variables; the first is ChatGPT text generators, the second is descriptive writing, and the third is grammar use.
Computer based adaptive testing in the early 1960s formed the initial use of adaptive learning which was developed to generative models and chatbot applications. Chat Generated Pretrained Texts (ChatGPT) is one of the generative models that are powered by Artificial Intelligence (AI) which generate human-like texts and other formats such as pictures. There has been a challenge in distinguishing between machined generated texts and human written texts. This challenge highlights the fast and accurate development of these chatbot and their uses in language education.
In his study, Hong (2023) aimed at elucidating the mechanisms, functions and common misconceptions about ChatGPT in education and learning. He stressed the various concerns that ChatGPT brought to education and learning. The study discussed the issues and risks that it can add to education. One of the challenges is that it is far from cognition and real language production compared to real practices. Second, it is not possible to ban it outside classes or campuses due to its availability. Third, the challenges related to validity and reliability issues of the generated texts. On the other hand, the benefits included using it as an effective tool for learning, and supplying teaching. It is also used as an authentic language tool and a personal language tutor. The study concluded that it is argued that ChatGPT offers major opportunities in EFL / ESL contexts.
In his discussion for the human-like text generation, Shijaku1 and Canhasi (2023) stated that "Generative models, such as ChatGPT, have gained significant attention in recent years for their ability to generate human-like text. However, it is still a challenge to automatically distinguish between text generated by a machine and text written by a human". (p.1). In his study, Frutos (2014) analyzed the development of LSP (Language for Specific Purposes) and the introduction of technology since 1960s; he concluded that text generators tools take into account contrastive analysis, genre rhetoric, and corpus linguistics. Further, text generators can be used as effective models for task based activities in EFL classes. Further, Zhai (2022) assured different concerns about the use of AI. First is bias due to the pre-trained input data. The second is Privacy as these machines process various students' data. Third is replacement of human jobs and the last is lack of transparency. Pretrained Text Generation is widely used in TEFL contexts. There have been different studies that tracked its development, highlighted its pros and discussed its cons. Educators are now concerned about how to design language settings and to implement learning environments that support FLL using ChatGPT tool.
Defining descriptive writing stems from the nature of this writing genre which focuses on word picturing. One of the two early definitions of descriptive writing was presented by Decker (1966) and Wyrick (1978). As for Deckers' definition, he defined it as "a form of writing that paints a word picture of something concrete, such as a scene or a person" (p. 23). Wyrick (1978) defines descriptive writing as "a form of writing that creates a word picture of persons, places, objects, and emotions using a careful selection of details to make impression on the reader' (p. 42). These early definitions shaped the wary EFL theory and practice viewed the skills related to descriptive writing as stating and providing information rather than analyzing or expressing views.
There are different studies in EFL literature that aimed to develop students' descriptive writing using various techniques, strategies and approaches (e.g. Sujarwaty, 2005; Rafika, 2014; Badawi, 2016). For instance, Badawi investigated the effectiveness of a program in enhancing EFL students' vocabulary knowledge and descriptive writing performance. Participants' number was 84 EFL majors ate October 6 University who were equally divided into control and experimental groups. The study concluded that the suggested program was effective in developing the dependent variables of the study including descriptive writing performance. Using senses in descriptive writing skills' development was highly recommended by the study.
In his study, Gebril (2001) investigated the effect of a program for developing EFL majors' descriptive writing skills. The study used the training program over an experimental group (N. 34) compared to a control group (N.43). Using a descriptive writing test, results showed a development in the experimental group's participants compared to the control group participants. The study also recommended focusing on EFL students' knowledge of descriptive writing skills and integrating different trainings for their development.
Abdullah (2022) developed descriptive writing skills of EFL students at the Faculty of Languages in MUST University using project based learning. The study had two-fold aim: the first was to identify the effect of the program using project based learning and the second was to identify the effect of the program on students' attitudes towards PBL. Participants (N. 80) were divided into two groups (control & experimental) equally. Instruments included a writing test and a questionnaire. Results showed that the program was effective in developing experimental group participants' descriptive writing skills. It also showed developments in their positive attitudes towards PBL.
There are different studies that integrated AI in teaching EFL skills including writing skills development. In her study, Zitouni (2022) implemented AI Lens tools of Artificial Intelligence for developing EFL skills of university students and measuring teachers' opinions about implementing AI tools in EFL classes. The study highlighted the pros and cons of utilizing AI tools in learning at university level including the salient role of AI tools in learning and improving the quality of e-learning. Abdalkader (2022) utilized AI for developing writing skills at pre-university level. The study aimed to identify the effect of using AI based activities on enhancing EFL writing fluency for the preparatory stage students. Results also showed the development in the participants' witting performance and the effective use of AI tools in writing skills development.
Descriptive writing is a key writing genre in TEFL research that is characterized by its distinctive features. It seeks to describe word-picture of characters, things, events, and emotions. Literature included various studies that aimed to integrate various approaches to develop various descriptive writing skills.
Grammar use mainly aims to effective communication with the language. In literature, grammar related studies focused on various aspects of grammar use, achievement, and development (e.g El Kenany, 2021; Soweilam, 2020; Badr & Ibrahim, 2022; Hani Bani, 2014)). Grammar as a key language area has been investigated in studies at both pre-university and at university level as well using different types of modules, programs and compiled units in different EFL contexts.
In his study, El Kenany (2021) highlighted the significance of teaching grammar and helping students to use it correctly. He further stated that "Grammar plays an important role in language teaching. Grammar helps students speak English with organized and correct structures and thus develop the communicative skills of the language" (p.61). The study focused on developing grammatical achievement and utilized mind mapping as an effective strategy in developing students'' grammar. Similarly, grammar was investigated by Soweilam (2020) using a different approach through utilizing task complexity when associated with recasts. The study assured that using interactive tasks can develop participants' English grammar use.
Using technology for developing English grammar use was investigated in some studies within different EFL contexts. For instance, Kim (2020) investigated the effects of artificial intelligence Chatbots on improving college students' English grammar skills. The study assured the positive impact of Chatbots in developing English grammar use. Similarly, Taguma et al., (2018) utilized Grammarly as a powerful AI based tool to enhance students' writing and grammar. The study used games coined with Grammarly application to create simulated environments to survive using words, structures and AI based activities during the e-games in the study. AI based technology h s positive impact on developing English grammar use at various EFL contexts.
Results of the study
Using SPSS statistical analysis, the following obtained quantitative results from the pre and post testing procedures verified the following hypotheses of the study:
H.1. There would be statistically significant mean differences between the scores of the participants in the pre and post testing of descriptive paragraph writing skills favoring the post testing.
To test this hypothesis, the following statistical results show the mean differences between the scores of the participants in the pre and post testing of their descriptive paragraph writing skills.
|
Table 1. Paired Samples Statistics
|
||||||||||||||||||||||
The above table (1) shows that the mean of the participants in descriptive paragraph writing skills was (47.53) in the pre testing compared to the post testing (70.03). This shows that there was an increase in participants' means in writing. The following table (2) compares between the means of the participants and shows the degree of significance between the pre and post testing procedures.
Table2. Paired Samples Test
|
|
|
Paired Differences |
t |
df |
Sig. (2-tailed) |
||||
|
|
Variables |
Mean |
Std. Deviation |
Std. Error Mean |
95% Confidence Interval of the Difference |
||||
|
|
|
Lower |
Upper |
||||||
|
Pair 1 |
Descriptive_writing_pre - Descriptive_writing_post |
-22.500- |
12.522 |
2.286 |
-27.176- |
-17.824- |
-9.841- |
29 |
.000 |
Table (2) shows that there are statistically significant mean differences between the participants in the pre and post testing of their descriptive paragraph writing skills (.000) favoring the post testing procedure. This shows that the performance of the participants was better in the post testing due to the effect of the program over the participants. To identify the effect size, the following calculation was run to identify the eta square that represents the effect size.
|
Table3. Effect size using Eta Square |
||||||
|
Source |
Type III Sum of Squares |
df |
Mean Square |
F |
Sig. |
Partial Eta Squared |
|
Corrected Model |
2383.050a |
13 |
183.312 |
.858 |
.605 |
.411 |
|
Intercept |
56279.770 |
1 |
56279.770 |
263.419 |
.000 |
.943 |
|
Descriptive_writing_post |
2383.050 |
13 |
183.312 |
.858 |
.605 |
.411 |
|
Error |
3418.417 |
16 |
213.651 |
|
|
|
|
Total |
73584.000 |
30 |
|
|
|
|
|
Corrected Total |
5801.467 |
29 |
|
|
|
|
|
a. R Squared = .411 (Adjusted R Squared = -.068) |
|
|
|
|||
The above obtained Eta Square value shows the effect size of the program on the participants (.411) as obtained by partial eta squared statistic. This shows the large size effect of the program based on ChatGPT in developing 3rd year EFL majors' descriptive paragraph writing skills.
H.2. There would be statistically significant mean differences between the scores of the participants in the pre and post testing of their English grammar use favoring the post testing.
To verify the above hypothesis, the following table shows the differences in means in the pre testing of English grammar use (33.00) compared to the post testing (50.40). This shows that the mean scores of the participants is higher after the application of the ChatGPT-based program compared to the pre-testing before applying the program.
|
Table 4. Paired Samples Statistics |
|||||
|
|
Variables |
Mean |
N |
Std. Deviation |
Std. Error Mean |
|
Pair 1 |
Grammar_pre |
33.00 |
30 |
12.126 |
2.214 |
|
Grammar_post |
50.40 |
30 |
11.628 |
2.123 |
|
The following table (5) compares between the means of the participants and shows the significance value (.000) due to these differences. This difference is clearly significant between the two testing procedures and this shows the impact of the program on English grammar use of the participants. The following statistical treatment was run to verify the hypothesis related to English grammar use of the participants.
Table5. Paired Samples Test
|
|
|
Paired Differences |
t |
df |
Sig. (2-tailed) |
||||
|
|
Variables |
Mean |
Std. Deviation |
Std. Error Mean |
95% Confidence Interval of the Difference |
||||
|
|
|
Lower |
Upper |
||||||
|
Pair 1 |
Grammar_pre - Grammar_post |
-17.400- |
11.331 |
2.069 |
-21.631- |
-13.169- |
-8.411- |
29 |
.000 |
The above table shows that there were significant means differences (0.000) between the participants in the pre and post testing of their English grammar. This assures that the program of the study could provide them with chances to develop their English grammar use. The following statistical running identifies the size of this effect. The following calculation was run to identify the eta square that represents the effect size.
|
Table6. Effect size using Eta Square |
||||||
|
Source |
Type III Sum of Squares |
df |
Mean Square |
F |
Sig. |
Partial Eta Squared |
|
Corrected Model |
2864.000a |
17 |
168.471 |
1.444 |
.262 |
.672 |
|
Intercept |
27110.778 |
1 |
27110.778 |
232.378 |
.000 |
.951 |
|
Grammar_post |
2864.000 |
17 |
168.471 |
1.444 |
.262 |
.672 |
|
Error |
1400.000 |
12 |
116.667 |
|
|
|
|
Total |
36934.000 |
30 |
|
|
|
|
|
Corrected Total |
4264.000 |
29 |
|
|
|
|
|
a. R Squared = .672 (Adjusted R Squared = .207) |
|
|
|
|||
The above table shows that the effect size was (.672) which refers to the significant effect for the program the participants. This effect value assures that the compiled program is effective and could develop the participants' English grammar use. The following section discusses these obtained results within the framework of pertinent literature and related studies. The following section discusses these obtained results in the light of the pertinent literature.
Discussion
The obtained results showed that there were significant mean difference between the participants in the pre and post testing of the descriptive paragraph writing skills and their grammar use favoring the post testing. The following discussion relates these results to literature and related studies.
ChatGPT based programs related to the writing skills proved to be effective in various sub-writing skills. For instance, the obtained results from this study agree with Zohery (2023) who explored ChatGPT use in academic writing skills and publishing stage. The study confirmed that ChatGPT is an effective tool in scientific writing in general and in writing areas such as paragraphing, summarizing, editing and proofreading in particular. He further confirmed the positive impact of these programs on both the quality and performance of the students, which was similarly resulted by the main study. He further concluded the advantages of using ChatGPT in writing to include speed, creativity, feedback, and engagement.
Similarly, Abu El Magd (2022) explored text Chatbots on EFL writing skills among Computer and Informatics students. Although the study dealt with writing but it was different in addressing technical writing while the present study dealt with different genre "descriptive writing". The study was similar in stressing the effectiveness of Chatbots in developing different writing skills. The results obtained from the study assure the positive impacts of Chatbots which included credibility, authenticity and being beneficial in their outcomes.
Differently, a study by Hong (2023) explored the impact of using ChatGPT on foreign language teaching and learning. It compared between students and teachers' uses of ChatGPT. Although the study agrees with the results of this main study in terms of the positive impacts of ChatGPT on students' writing, it further stressed the same impacts on teachers' use of this text generator genre. The study also assured that ChatGPT can provide the institutions and the teachers with different opportunities to improve their students' writing and assessment.
In a different context, a study by Guo, Wang & Chu (2022) identified the effectiveness of using Chatbots to scaffold EFL students' argumentative writing. Although the writing skills addressed in the study are different but the results agree with the obtained results from this study in highlighting the value of text generators in EFL classes. The results assured the development of students' argumentative writing skills and could result high quality argumentative writing. The study agrees with the limitations of text generator model of the study in its retrieval model rather than generative model. The outputs obtained are pre-defined rather than newly generated. The present study confirms the limitation of ChatGPT in providing feedback to students' writing and skill development.
Implications of the study
Implications of the study stem from the challenges faced during its running. The first challenge is the logging in challenge; there were many crashes during texts generation and processing due to the user issues and the region limitations. Participants also utilized ChatGPT for answering and completion of the questions of the tests. They were asked not to use it for complete answer and to perform the tasks as required during the testing and administration phases. The study can be useful for many:
First, students can make use of the study program in enhancing their writing skills and their use of English grammar. They can adapt the activities in the course for various language practices and different areas of language. They can use index in the program for better grammar practice and in their error correction.
Second, instructors at university level can use the program in their writing and grammar classes. The course can be integrated in the course description and can be uploaded as an electronic resource for the students on the course Moodle. Teachers at both pre university levels can also make use of the materials in the course to develop both writing and grammar oriented skills.
Third, course designers can also make use of the program in designing similar programs or integrating the scope and sequence of the program in other courses. The framework of the course can be useful in compiling e-courses for developing writing and grammar using other Ai-based tools.
Recommendations and suggestions for further research
In the light of the obtained results, the study recommends integrating ChatGPT in EFL context. It can be useful in developing the four skills, vocabulary and English grammar use. There is a need to design similar programs using ChatGPT for developing other language areas such as translation. Using the programs based on ChatGPT can be integrated in EFL classes partially or wholly as set courses in different university grades. The study calls for integrating AI-based apps and tools in EFL courses' description as useful tools to be integrated during language skills practice such as critical reading, summarizing, and paraphrasing. The study also recommends rethinking of assessing the productive skills (writing and speaking) using Ai-based technology; using AI-based rubrics can save the time and ensure the accuracy of feedback.
The study suggests conducting research that investigates the effect of using AI-based tools in enhancing receptive skills (listening and reading. The study also calls for integrating text generators in essay writing skills in general and descriptive essays in particular. Conducting a research that integrates ChatGPT in academic writing skills development is also recommended. In the light of the obtained results, the study also suggests running a research about enhancing translation skills through using ChatGPT for concepts and idioms translations; it can also be used for sentence and text translation as well as performance-based assessment.
Conclusions
There have been positive impacts for using text generators such as ChatGPT tools in English as a Foreign Language context. The present study is one of the studies that aimed to identify the effect of a program based on ChatGPT text generators in enhancing descriptive paragraph writing and English grammar use of EFL Majors in the Egyptian context. The obtained results from the tests of the study assured the effectiveness of using ChatGPT as a technological tool that develops writing and grammar. These results confirm the distinctive characteristics of text generators that include authenticity, value, and integration apart from its reliability as a source input. Using AI based technology for language development should be integrated in EFL course design as well as course description. This technology can be integrated in both course activities and assessment. Using these tools in assessment can be useful in productive skills assessment by using effective AI generated rubrics. The study adds to the theory and practice of using a controversial tool (ChatGPT) in English as a Foreign Language EFL context.
References
Abdalkader, S. (2022). Using Artificial Intelligence to improve Writing Fluency for The Preparatory Stage Students in Distinguished Governmental Language Schools. Egyptian Journal of Educational Sciences, 2 (2), pp. 40 -70.
Abdullay, E. (2022). The effect of a problem based approach PBL on EFL students' descriptive writing approach. Egyptian Knowledge & Reading Journal. 12(254), pp. 1-15.
Abo Laban, M. & Firwana, S. (2017). The effectiveness of using mobile learning in developing eleventh graders' English grammar learning and motivation for English. Dissertation, Islamic University, Palestine.
Abu El-Magd, M. (2022). Text Chatbots Assisted Edublogs for enhancing the EFL technical writing performance among computer and informatics students. Journal of Faculty of Education, 2 (46).
Azaria, A. (2022). ChatGPT Usage and Limitations. Retrieved 19th July, 2022 from https://www.researchgate.net/publication/366618623_ChatGPT_Usag
Badr, B. & Ibrahim, D. (2022). Using discourse based grammar teaching to reduce grammatical errors in English language majors' writing. Journal of Faculty of Education, Ain Shams University, 2 (26).
Badwai, M. (2016). A sensory-based program for developing EFL major's sensory vocabulary knowledge and descriptive writing performance. Arab Studies in Education and Psychology, 71 (1), 527-468.
Chinonso, O. (2023). Chatbot for teaching, learning and research: prospects and challenges. Retrieved: https://www.researchgate.net/publication/366808899
Decker, R. (1966). Patterns of Exposition. Little, Brown & Co.
Demir, E. &Zaremba, A. (2023). ChatGPT: Unlocking the Future of NLP in Finance. Retrieved: SSRN: https://ssrn.com/abstract=4323643 or http://dx.doi.org/ 10.2139/ssrn.4323643
El-Kenany, D. (2021). Improving EFL Grammar Achievement through Using Electronic Mind Maps. Journal of The Faculty of Education- Mansoura University No. 113.
El Remali, A. (2017). Teaching English grammar to Libyan university students: current situations and suggestions to improve. Libyan International Journal,20, pp. 1-8.
Frutos, A. (2014). Text Generators: an aid for writing in the tertiary EST class. CORELL: Computer Resources for Language Learning, 4, 27-41
Gawate, S. P. (2019). Artificial Intelligence (AI) Based Instructional Programs in Teaching Learning of English Language. International Journal of English Language, Literature and Translation Studies (IJELR), 6, 6.
Gebril, A. (2001). The effect of a suggested program on developing first year EFL majors' descriptive writing skills at Sohag Faculty of Education. MA Thesis, Sohag Faculty of Education, Sohag University.
Guo, U.; Wang, J.; Chu, S. (2022). Using Chatbots to scaffold EFL students' argumentative writing. Assessing Writing. 45 (1)
Hakmy, H. & Abdel Rahman A. (2019). The effects of using games in teaching and learning English grammar in EFL learners. Educational and Psychological Sciences Journal, 3 (26), pp. 147-159.
Han, J. & Kim, N. (2016). The effects of post-tasks CMC activities and task types on Korean EFL learners’ oral performance. STEM Journal, 17(2), 109-135.
Hani, A. (2014). The effect of using computer in teaching English grammar to the Jordanian university students: the passive voice a model. Journal of Education and Psychological Sciences, 15 (4).
Haristiani, N., & Danuwijaya, A. A. (2019). Gengobot: A chatbot-based grammar application on mobile instant messaging as language learning medium. Journal of Engineering Science and Technology, 14(6), 3158-3173.
Hong, W. (2023). The impact of ChatGPT on foreign language teaching and learning: Opportunities in education and research. Journal of Educational Technology and Innovation, 3 (1).
Kengam, J. (2020). Artificial intelligence in education. Research Gate.
Kim, H., Young, N. & Cha, Y. (2021). Is it beneficial to use AI Chatbots to improve learners' speaking performance? The journal of Asia TEFL, Doi:http://doi.org/20.18823/2021.18.1.10.161
Kim, H., Young, N. & Cha, Y. (2021). Effects of AI Chatbots on EFL Students’ Communication Skills. Korean Journal of English Language and Linguistics, 21(1), pp. 712-734
Liu, M. (2023). Exploring the Application of Artificial Intelligence in Foreign Language Teaching: Challenges and Future Development. CLEC, SHS Web of Conferences, 168(1), 03025. DOI: https://doi.org/10.1051/shsconf/202316803025
Nutta, J. (2001). Is computer based grammar instructions effective as teacher directed grammar instruction for teaching L2 structures. USA: University of South Florida.
Rafika, M. (2014). Teaching descriptive text writing through guided Wh-questions: A pre-experiment student at the eighth grade students of SMP Negeri 7 pontianak in academic year 2013 – 2014, WKS, Studies in English Language and Education.1 (1), pp. 51-58
Saleh, Z. (2019). Artificial Intelligence Definition, Ethics and Standards. Journal of Artificial Intelligence,1(1).
Samir, S. & Firwana, S. (2015). The effect of using jigsaw strategy on Palestinian tenth graders' English grammar reading. MA Thesis, the Islamic University, Palestine.
Shijaku1, R. & Canhasi, E. (2023). ChatGPT Generated Text Detection. Retrieved: https://www.researchgate.net/publication/366898047
Soweilam, E. (2020). The effect of employing task complexity with recasts on improving grammar acquisition among EFL Egyptian learners. Journal of Scientific Research in Education,11 (21). DOI: 137570.2021.JSRE/21608.1
Stephens, C. (2023). Grammar, usage & Style. Retrieved: https://study.com/academy/subj/english.html.
Sujarwaty, I. (2005). The influence of using Wh-questions as teaching guidance towards the students' ability in writing recount text. Journal of Education, 5(11), 106-121
Wang, Y., & Petrina, S. (2013). Using learning analytics to understand the design of an intelligent language tutor – Chatbot Lucy. Editorial Preface, 4(11), 124-131.
Wyrick, J. (1978). Steps to Writing Well. Cengage Learning Prints.
Zhai, X. (2022). ChatGPT User Experience: Implications for Education. Retrieved: https://orcid.org/0000-0003-4519-1931
Zitouni, K. (2022). New trend in EFL online learning and teaching through the lens of artificial intelligence. Almqadimah of human & Social studies,7 (1), pp. 1065-1085
Zohery, M. (2023). ChatGPT in academic writing and publishing: A comprehensive guide. Retrieved: Doi:10.5281/zenodo.7803703
', './data/mfes/coversheet/971693641577.jpeg'))
مشاهدة على SCiNiTO