ChatGPT: The End of Online Exam Integrity?

Simple item page
dc.citation.issue6
dc.citation.volume14
dc.contributor.authorSusnjak T
dc.contributor.authorMcIntosh TR
dc.contributor.editorMuijs D
dc.date.accessioned2024-07-24T03:13:45Z
dc.date.available2024-07-24T03:13:45Z
dc.date.issued2024-06-17
dc.description.abstractThis study addresses the significant challenge posed by the use of Large Language Models (LLMs) such as ChatGPT on the integrity of online examinations, focusing on how these models can undermine academic honesty by demonstrating their latent and advanced reasoning capabilities. An iterative self-reflective strategy was developed for invoking critical thinking and higher-order reasoning in LLMs when responding to complex multimodal exam questions involving both visual and textual data. The proposed strategy was demonstrated and evaluated on real exam questions by subject experts and the performance of ChatGPT (GPT-4) with vision was estimated on an additional dataset of 600 text descriptions of multimodal exam questions. The results indicate that the proposed self-reflective strategy can invoke latent multi-hop reasoning capabilities within LLMs, effectively steering them towards correct answers by integrating critical thinking from each modality into the final response. Meanwhile, ChatGPT demonstrated considerable proficiency in being able to answer multimodal exam questions across 12 subjects. These findings challenge prior assertions about the limitations of LLMs in multimodal reasoning and emphasise the need for robust online exam security measures such as advanced proctoring systems and more sophisticated multimodal exam questions to mitigate potential academic misconduct enabled by AI technologies.
dc.description.confidentialfalse
dc.edition.editionJune 2024
dc.identifier.citationSusnjak T, McIntosh TR. (2024). ChatGPT: The End of Online Exam Integrity?. Education Sciences. 14. 6.
dc.identifier.doi10.3390/educsci14060656
dc.identifier.eissn2227-7102
dc.identifier.elements-typejournal-article
dc.identifier.number656
dc.identifier.urihttps://mro.massey.ac.nz/handle/10179/70317
dc.languageEnglish
dc.publisherMDPI (Basel, Switzerland)
dc.publisher.urihttps://www.mdpi.com/2227-7102/14/6/656
dc.relation.isPartOfEducation Sciences
dc.rights(c) 2024 The Author/s
dc.rightsCC BY 4.0
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectAI impact on education
dc.subjectChatGPT in exam cheating
dc.subjectmultimodal online assessments
dc.subjectGPT-4 vision evaluation
dc.subjecteducational integrity
dc.subjectgenerative AI
dc.subjectexamination security
dc.subjectlarge language models in assessments
dc.subjectcritical thinking in LLMs
dc.subjectmulti-hop reasoning in LLMs
dc.subjectmultimodal chain-ofthought reasoning
dc.titleChatGPT: The End of Online Exam Integrity?
dc.typeJournal article
pubs.elements-id489648
pubs.organisational-groupCollege of Health
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Published version.pdf
Size:
554.54 KB
Format:
Adobe Portable Document Format
Description:
489648 PDF.pdf
Download
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
license.txt
Size:
9.22 KB
Format:
Plain Text
Description:
Download
Collections
Journal Articles