Can AI ace exams?

Faculty across fields have been testing out the new ChatGPT website and sharing the surprisingly sophisticated results the site can generate using its machine-learning trained language model. Many are questioning whether this will be the end of take home exams, since students could very easily copy and paste answers from this freely-accessible site. I’m not an expert on technology, much less AI, and a lot of the explainers out there assume more tech knowledge than most faculty have. So, based on my reading (subject to editing and improvements), here are some basic Q&A’s for faculty wondering how this may change the game–as early as this year’s final exams.

What kinds of questions can ChatGPT answer?

It isn’t just a Q&A tool; it can also choose the correct option for a multiple choice question, write or debug code, generate creative writing including parodies, and provide instructions. Because it draws on a vast amount of data and is a machine-learning engine, it can even answer questions that require you to apply obscure knowledge to broader questions or hypothetical situations.

It can also generate multiple, slightly different answers to the same question if you ask it to try again.

Does ChatGPT give good answers?

It gives answers that sound plausible and even sophisticated, but those answers often lack nuance and are sometimes incorrect. In some cases, the answers may even be nonsensical, while in others, they are quite good. If bias exists in the corpus of material it draws on, the AI will reproduce that bias.

Many faculty who have shared their experiences testing the site with their own exam questions say the answers they’re getting are in the B to C grade range. Scroll down for sample questions and answers.

Is there any way to check to see if a student has used ChatGPT to answer a question?

This AI detector has had mixed results. Note that it was built to detect an earlier version of the AI model than ChatGPT uses.

How do you write questions that ChatGPT can’t answer?

It’s pretty hard to stump it entirely, but you can establish criteria for your answer that make it impossible for students to use it. You might require citations, particularly to in-class readings or lectures, or for less open-ended questions, you might require the student to explain how they came to their answer. You could also shift to assignments that require students to apply their knowledge through the use of art, technology, or other media.

Can ChatGPT be used for teaching?

The inherent faults of this site could, some faculty have suggested in posts on social media, become teaching tools. You could post a question with ChatGPT’s answer and ask students to offer critiques to create a better answer. Students might also be asked to grade multiple ChatGPT answers and explain the grades.

Amazingly, ChatGPT can also create assignment prompts and rubrics according to the parameters you suggest. This article describes many ways faculty could have ChatGPT help with the work of teaching.