Duration: 20 Feb 2024 → 27 Feb 2024
Conference number: 38th
(AAAI-24 Technical Tracks 13)
(AAAI-24 Technical Tracks 14)
(AAAI-24 Technical Tracks 18)
(Website)
Name | Proceedings of the AAAI Conference on Artificial Intelligence |
---|---|
Publisher | Association for the Advancement of Artificial Intelligence (AAAI) |
Number | 20 |
Volume | 38 |
ISSN (Print) | 2159-5399 |
ISSN (Electronic) | 2374-3468 |
Conference | AAAI Conference on Artificial Intelligence 2024 |
---|---|
Abbreviated title | AAAI 2024 |
Country/Territory | Canada |
City | Vancouver |
Period | 20/02/24 → 27/02/24 |
Internet address | (AAAI-24 Technical Tracks 13) (AAAI-24 Technical Tracks 14) (AAAI-24 Technical Tracks 18) (Website) |
T1 - Towards automatic boundary detection for human-AI collaborative hybrid essay in education
AU - Zeng, Zijie
AU - Sha, Lele
AU - Li, Yuheng
AU - Yang, Kaixun
AU - Gašević, Dragan
AU - Chen, Guanliang
N1 - Conference code: 38th
N2 - The recent large language models (LLMs), e.g., ChatGPT, have been able to generate human-like and fluent responses when provided with specific instructions. While admitting the convenience brought by technological advancement, educators also have concerns that students might leverage LLMs to complete their writing assignments and pass them off as their original work. Although many AI content detection studies have been conducted as a result of such concerns, most of these prior studies modeled AI content detection as a classification problem, assuming that a text is either entirely humanwritten or entirely AI-generated. In this study, we investigated AI content detection in a rarely explored yet realistic setting where the text to be detected is collaboratively written by human and generative LLMs (termed as hybrid text for simplicity). We first formalized the detection task as identifying the transition points between human-written content and AI-generated content from a given hybrid text (boundary detection). We constructed a hybrid essay dataset by partially and randomly removing sentences from the original studentwritten essays and then instructing ChatGPT to fill in for the incomplete essays. Then we proposed a two-step detection approach where we (1) separated AI-generated content from human-written content during the encoder training process; and (2) calculated the distances between every two adjacent prototypes (a prototype is the mean of a set of consecutive sentences from the hybrid text in the embedding space) and assumed that the boundaries exist between the two adjacent prototypes that have the furthest distance from each other. Through extensive experiments, we observed the following main findings: (1) the proposed approach consistently outperformed the baseline methods across different experiment settings; (2) the encoder training process (i.e., step 1 of the above two-step approach) can significantly boost the performance of the proposed approach; (3) when detecting boundaries for single-boundary hybrid essays, the proposed approach could be enhanced by adopting a relatively large prototype size (i.e., the number of sentences needed to calculate a prototype), leading to a 22% improvement (against the best baseline method) in the In-Domain evaluation and an 18% improvement in the Out-of-Domain evaluation.
AB - The recent large language models (LLMs), e.g., ChatGPT, have been able to generate human-like and fluent responses when provided with specific instructions. While admitting the convenience brought by technological advancement, educators also have concerns that students might leverage LLMs to complete their writing assignments and pass them off as their original work. Although many AI content detection studies have been conducted as a result of such concerns, most of these prior studies modeled AI content detection as a classification problem, assuming that a text is either entirely humanwritten or entirely AI-generated. In this study, we investigated AI content detection in a rarely explored yet realistic setting where the text to be detected is collaboratively written by human and generative LLMs (termed as hybrid text for simplicity). We first formalized the detection task as identifying the transition points between human-written content and AI-generated content from a given hybrid text (boundary detection). We constructed a hybrid essay dataset by partially and randomly removing sentences from the original studentwritten essays and then instructing ChatGPT to fill in for the incomplete essays. Then we proposed a two-step detection approach where we (1) separated AI-generated content from human-written content during the encoder training process; and (2) calculated the distances between every two adjacent prototypes (a prototype is the mean of a set of consecutive sentences from the hybrid text in the embedding space) and assumed that the boundaries exist between the two adjacent prototypes that have the furthest distance from each other. Through extensive experiments, we observed the following main findings: (1) the proposed approach consistently outperformed the baseline methods across different experiment settings; (2) the encoder training process (i.e., step 1 of the above two-step approach) can significantly boost the performance of the proposed approach; (3) when detecting boundaries for single-boundary hybrid essays, the proposed approach could be enhanced by adopting a relatively large prototype size (i.e., the number of sentences needed to calculate a prototype), leading to a 22% improvement (against the best baseline method) in the In-Domain evaluation and an 18% improvement in the Out-of-Domain evaluation.
KW - General
UR - http://www.scopus.com/inward/record.url?scp=85188603524&partnerID=8YFLogxK
U2 - 10.1609/aaai.v38i20.30258
DO - 10.1609/aaai.v38i20.30258
M3 - Conference Paper
AN - SCOPUS:85188603524
T3 - Proceedings of the AAAI Conference on Artificial Intelligence
BT - Thirty-Eighth AAAI Conference on Artificial Intelligence
A2 - Wooldridge, Michael
A2 - Dy, Jennifer
A2 - Natarajan, Sriraam
PB - Association for the Advancement of Artificial Intelligence (AAAI)
CY - Washington DC USA
T2 - AAAI Conference on Artificial Intelligence 2024
Y2 - 20 February 2024 through 27 February 2024
share this!
June 18, 2024
This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:
fact-checked
trusted source
by Kobe University
To get high scores on essay writing tests, learners of English as a foreign language need to focus on good arguments more than on complex grammar. The Kobe University finding challenges conventional approaches to test preparation and scoring rubrics.
Writing essays is a well-established tool for monitoring progress in learning English as a foreign language , as it provides a snapshot of a student's mastery of grammar and vocabulary. Especially in Japan, where English language tests are often required for university admission and students closely follow advice on how to achieve high scores on these tests, a "good essay " is often seen as one that demonstrates a high level of grammatical complexity. But is this actually reflected in test scores ?
Kobe University linguist Yasuda Sachiko says, "Based on my experience of teaching academic writing to students at various levels in Japan, I believe that linguistically complex texts do not always result in better writing."
She therefore decided to conduct an experiment with over 100 Japanese high school students. Yasuda had them write a short essay on a given topic and looked at the relationship between the linguistic complexity of the texts and the writers' ability to present complex arguments, and how these two related to how the texts were graded according to official rubrics.
She adds, "This study is the first to focus on the relationship between features of linguistic complexity and features of meaning complexity; no one else in the relevant fields has looked at the relationship between these two."
The results, published in the journal Assessing Writing , confirmed her suspicions. She found that high-scoring essays shared features related more to the ability to express complex meaning, such as lexical diversity, noun modification, and soundness and number of arguments, than to structural complexity.
"Interestingly, low scoring essays showed the highest level of complexity in finite adverbial dependent clauses," the linguist writes in her paper. Emphasizing this point, the ability to express complex meaning was strongly correlated only with using diverse expressions and the ability to modify their meaning, but not with grammatical features.
Yasuda concludes, "Simply having complex sentence structures does not necessarily lead to a better essay."
The findings have implications for how essay writing tests are scored. The Kobe University researcher explains, "Current rubrics for writing questions on language tests instruct test-takers to 'use complex grammar appropriately' or 'a variety of complex structures."
However, since sentence complexity does not significantly affect overall essay quality, it may be more appropriate to use terms such as 'contextually appropriate grammar' or 'genre-appropriate grammar.'
Thus arguing that the ability to express one's opinion in varied and complex ways is a marker of students' writing ability, she advocates that this characteristic should be more represented both in the way tests are scored and how feedback is provided to students.
This so-called washback effect of test scoring rubrics on the way language is taught is at the heart of what drives Yasuda.
She says, "I am committed to using the results of this study for practical applications, such as refining assessment criteria for evaluating students' writing, developing tasks and materials to improve their writing skills, and identifying the key knowledge that teachers need to help students become better writers.
"The ability to write in English has become increasingly important in the 21st century, as it is a crucial medium that allows us to connect with others around the world."
Provided by Kobe University
Explore further
Feedback to editors
18 hours ago
23 hours ago
Jun 21, 2024
Relevant physicsforums posts, how is physics taught without calculus.
Jun 20, 2024
Jun 16, 2024
May 27, 2024
May 16, 2024
May 13, 2024
May 11, 2024
More from STEM Educators and Teaching
May 8, 2024
May 9, 2023
Jun 9, 2021
Jul 10, 2023
Dec 15, 2017
Apr 21, 2022
Jun 10, 2024
May 2, 2024
Apr 18, 2024
Apr 17, 2024
Apr 8, 2024
Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).
Please select the most appropriate category to facilitate processing of your request
Thank you for taking time to provide your feedback to the editors.
Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.
Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Phys.org in any form.
Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.
More information Privacy policy
We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.
units_index
/pubs/2019handbooks/units/index-byarea-of-study-w-writing.html
IMAGES
COMMENTS
Writing a body paragraph. The body of the essay is where you fully develop your argument. Each body paragraph should contain one key idea or claim, which is supported by relevant examples and evidence from the body of scholarly work on your topic (i.e. academic books and journal articles).
An academic essay is a piece of writing in which you present your position on a topic, and support that position by evidence. An essay has three main parts: introduction, body, and conclusion. In the introduction, you put forward your position (this can take the form of a question or an argument) and its relevance to the chosen topic. In the ...
Example essay outlines. Below are two examples of essay outlines that were written in response to the essay question: 'Explain the relationship between police culture and police accountability'. Version 1 reflects the sort of plans that many students produce. While it works as a starting point, it needs to be further developed.
Develop your writing skills for any assessment or task, learn new writing strategies and master academic writing . ... an essay; an infographic; a journal article; a literature review; an oral presentation; a poster; a reflection; ... Monash University: 00008C Monash College: 01857J.
See a model of the basic physical structure of an academic essay. In the body of your essay, clarify your position and and develop and support your argument. The body contains the content of your essay. Focus on answering the question and providing evidence in the body paragraphs. Summarise main points/ highlight key findings in the conclusion.
Synopsis. This unit introduces students to academic writing and the attitudes, values and practices of academic culture. The unit will help students to write effective essays at a tertiary level. There is a particular focus on how academic writing presents clear reasoning with evidence to support a position, and how the academic essay differs ...
This unit introduces students to academic writing and the attitudes, values and practices of academic culture. The unit will help students to write effective essays at a tertiary level. There is a particular focus on how academic writing presents clear reasoning with evidence to support a position, and how the academic essay differs from other ...
Overview. This unit will introduce you to the requirements of writing for academic purposes. It will provide you with a range of composition skills that are essential for producing academic essays, reports and reviews. The unit will introduce writing styles and conventions appropriate to university study and provide you with practical insights ...
Synopsis. This unit introduces students to academic writing and the attitudes, values and practices of academic culture. The unit will help students to write effective essays at a tertiary level. There is a particular focus on how academic writing presents clear reasoning with evidence to support a position, and how the academic essay differs ...
It contains dedicated chapters on writing an essay, a reflective text, a case study analysis, a literature review, a placement report, and case notes on placement. Each chapter contains examples and activities which will help students to test their knowledge and understanding. This is an essential companion for all Social Work students.
The Monash Editorial Style Guide governs writing conventions for the University's website, official social media channels, emails, and print publications. Many people first experience Monash via the pages they seek out or stumble across on our website. Because we can't control where our users enter our site, all our content must meet the same ...
Week 1 and 2. Take our interactive tutorial about academic integrity. Learn essential research skills in Library workshops. Workshops are run as part of first-year Law units for both undergraduates and postgraduates. Read the resources on your unit Moodle sites, which include important information from your lecturers about the units you'll be ...
Sample Literary Studies essay. This is a real student's essay. The essay is good, however it does have some problems that have been addressed in the notes accompanying it. Read through the essay and the feedback notes in the boxes on the right to see its weaknesses and strengths. Remember that there is no right or wrong with writing essays, but ...
These are: In-text referencing: where the Author and Year of publication are identified in the essay and a list of References which have been cited are placed at the end of the essay. Examples of this style are Monash Harvard; APA; MHRA; Chicago and MLA. Footnote referencing: where a number is allocated to each reference which is usually listed ...
The Writing minor will introduce you to a range of writing practices in the context of communication and media studies, cultural theory, and literary studies. You will become familiar with conventions and experimentation in contemporary writing, especially prose forms, and gain appreciation for the various techniques associated with them in ...
Getting started . Monash University Graduate Education - Monash Doctoral Program See the Library's Graduate Research Library guide for information.. Use EndNote or Zotero to keep track of your references and format footnote citations in your document.. See Managing References and Notes (RLO) for more information.. Find out about Storage and Backup guidelines.. If you can't find a resource at ...
Minimum total expected workload equals 144 hours per semester comprising: Contact hours for on-campus students: workshops and lectures. tutorials. online activity. Additional requirements: independent study to make up the required minimum hours during the semester. See also Unit timetable information. EDF1013: Academic writing - Monash University.
Using this guide. The Law research and writing guide introduces you to physical and online information, resources and tools. They have all been carefully selected to help you study or research Law at Monash. The guide will also help you to develop your skills in researching and writing for law. The home library for Law is the Law Library ...
It contains dedicated chapters on writing an essay, a reflective text, a case study analysis, a literature review, a placement report, and case notes on placement. Each chapter contains examples and activities which will help students to test their knowledge and understanding. This is an essential companion for all Social Work students.
Print MWRITING02 - Writing page. bookmark_border. MWRITING02 - Writing. info. sms_failed. There is a more recent version of this academic item available. ... Authorised by: Student and Education Business Services Monash University CRICOS Provider Number: 00008C Monash College CRICOS Provider Number: 01857J.
Abstract. Through a consideration of ways in which settler poets are unwriting their writing on Country, this essay asks how settler writing can function as an unsettling of ongoing colonial appropriations of Indigenous Country. The essay reads two poems by John Kinsella which he designates 'spatial concretions' or 'demappings' as forms ...
TY - GEN. T1 - Towards automatic boundary detection for human-AI collaborative hybrid essay in education. AU - Zeng, Zijie. AU - Sha, Lele. AU - Li, Yuheng
The Kobe University finding challenges conventional approaches to test preparation and scoring rubrics. To get high scores on essay writing tests, learners of English as a foreign language need to ...
Overview. In Introduction to Professional Writing, students will learn a range of genres in professional writing and their respective conventions from campaign writing and basic journalism to business writing and so forth. Students will first be taught the four main types of writing, but with particular focus on descriptive, expository ...
AMU2315 Strategies in writing experiments. AMU2448 Film genres. AMU2498 Contemporary fiction. AMU2690 Discovering the Asia-Pacific study trips. AMU2787 Multimedia journalism. AMU2832 Postcolonial and diasporic literature. AMU3499 Authorship and writing. AMU3575 Task force: Responding to global challenges.