Reading between the ChatGPT battle lines of students and educators

 What is ChatGPT?

The ‘GPT’ in ChatGPT stands for ‘generative pre-trained transformers’, a type of AI model that has gained immense popularity in natural language processing. ‘Generative’ signifies the model’s capability to generate a coherent text that is relevant to the context in question. ‘Pre-trained’ indicates that the model was trained to a large corpus of online text, where it learned to understand statistical patterns, grammar, and linguistic context.

Central to GPT is its ‘transformer architecture,’ deep neural networks (DNN) designed to process sequences of data, thereby priming them for neuro-linguistic programming (NLP). The transformer architecture involves attention mechanisms that allow the model to weigh the relevance of words in a sentence when making predictions.

GPT is designed to make use of ‘transfer learning’ principles. Having been trained on large volumes of text, that can be fine-tuned to fit in specific contexts, GPT models are then adapted, through transfer learning, to various applications: translation, summarization, answering questions, among many other tasks.

GPT models have proven strikingly successful at handling many NLP tasks, with applications ranging from chatbots and AI-based virtual assistants to text creation and completion tools. From GPT-1 to GPT-4, through GPT-3 and ChatGPT, GPT variants have taken large and rapid strides in the language generation journey, and spearheaded the ever-increasing development of generative AI.

 ChatGPT vs. students

The new study incorporates ChatGPT’s answers to a selection of diverse assessment questions on topics ranging from computer science, to political science, to engineering, to psychology, among many more disciplines. Researchers selected a representative sample of questions to assess ChatGPT’s knowledge and cognitive capabilities, compared to those of university students.

The researchers asked faculty members, who have collectively taught 32 various courses at NYUAD, to provide 10 questions from a course that they have taught.

Then, for each course, ChatGPT was used to generate three distinct answers to each of the 10 questions. Those answers were graded in comparison with the ones provided by three students.

The grades received by ChatGPT were, on average, comparable, or superior, to those received by students in 9 out of 32 courses.

In mathematics and economics, students were found consistently superior. In the case of ‘introduction to public policy’ course, ChatGPT vastly outperformed students, with an average grade of 9.56 received by the former, compared to 4.39 received by the latter.

 AI-fueled tension

Talal Rahwan, a computer scientist at NYUAD and co-author of the study, said: “The study makes clear that many students are willing to use this tool to help them with their assignments, even though many educators dismiss its use as outright plagiarism,” Rahwan told Nature Middle East.

To better understand the perspectives of students and educators on whether the AI-based tool should be used in academia, the authors surveyed 1601 participants, from five countries; Brazil, India, Japan, United Kingdom, and the United States.

According to the study, 74% of students indicated that they would use ChatGPT, while educators underestimated this share. On the other hand, 70% of the surveyed educators reported that they would treat its use as plagiarism.

Ethical dilemma

Since AI-based models are trained on immense amounts of data available on the internet, there is ongoing debate over the ownership of this data. Added to this is the growing difficulty of distinguishing between what’s human and what’s AI-generated. ChatGPT’s ability to write essays and generate solutions to assignments has raised questions over academic integrity violations by school and university students. In the United States, for instance, the tool has been banned by school districts in New York City, Los Angeles, and Baltimore. Similarly, Australian universities have announced their intention to return to hand-written exams to combat students using the tool for writing essays.

The study reports that many educators have voiced concerns regarding plagiarism, with professors from George Washington University, and Rutgers University, opting to phase out take-home, open-book assignments entirely.

In the realm of academic publishing, a number of conferences and journals have also banned the use of ChatGPT to produce academic writing. Yet, many have argued for its potential benefits as a tool for improving writing output, with some even pushing to reform the pedagogical underpinnings of the evaluative process in the education system. Using two AI-text detectors, GPTZero and OpenAI’s Text Classifier, the researchers found that the two tools misclassified ChatGPT’s answers as human-generated, by 32% and 49% respectively.

 Education disrupted

Speaking to Nature Middle East, Rahwan described the tension between students’ use of ChatGPT and educators’ propensity to dismiss its use as plagiarism. This can create conflict as students try to use the tool secretly, while educators strive to detect it.

The study findings suggest that ChatGPT could undermine the higher education institutions’ ability to validate data, and assess student knowledge and attainment. “It could also make educators’ feedback to students less effective,” Rahwan said. “It is critical to make sure that what you’re grading has been completed by the student who will receive the grades. Those grades affect access to scholarships and job opportunities, and even salaries in some countries and industries.”

“This tool represents a big challenge for the educational institutions struggling to develop academic integrity policies that account for the extensive applications of generative AI,” he added. “In particular, we believe that students’ ability to express ideas in their own words will continue to be an integral part of their learning journey. Consequently, to be totally dependent on this tool threatens their ability to write and express their ideas.”

Still, Rahwan is of the view that ChatGPT, and other generative AI tools, could help some students improve their writing skills. “Both educators and students agree that ChatGPT will boost the competitiveness of non-native English speaking students,” he said.

Sign-up to receive our e-alert update every two weeks to keep up with everything new on the portal.