It is supposed to be the next big thing to revolutionize education. Maybe at some point it will. But based on my students’ comments, ChatGPT/AI is less than a revolution.
At the beginning of 2023, ChatGPT/AI took higher education by storm. It was heralded as the next big thing to change teaching and learning. It was going to be the savior or demise of higher education as we know it. Yet as we know, the road to educational reform is littered with educational technology failures and overhypes.
In the fall 2023, I incorporated ChatGPT into my three undergraduate classes. For all three classes ChatGPT was used in a variety of ways. For each class, a ChatGPT question was listed in the syllabus for each day, serving as a prompt for discussion. I used ChatGPT to encourage student discussion. If I asked a question and no one was willing to answer I told the class to look it up in ChatGPT and tell me what it said.
I encouraged students in all three classes to use ChatGPT to help them generate preliminary answers for their take-home essay tests or papers. They could use it for research, to generate outlines, or for summaries of lectures. They could use it any way they wanted.
Finally, I generated several lesson plans on topics to be covered in class. I did this after already preparing my own so that I could compare them.
What did we learn?
My students were underwhelmed or generally unimpressed with ChatGPT as a teaching and learning tool. I chose not to use any of the ChatGPT generated lesson plans. They were simply not well organized, superficial, or got critical facts or issues wrong.
Several students remarked how they had been using it even before my classes. They found it mildly useful to generate some preliminary ideas for research or a paper, such as perhaps producing a preliminary outline that identified key issues or points. They were extremely critical that ChatGPT did not provide sources or references to books or articles. They all told me that they did not think ChatGPT did not do anything that Google did not already do, or the latter did better. Several said that using Google or another search engine along with ChatGPT helped them.
My students were highly aware and critical of the biases and inaccuracies in ChatGPT and how it makes up facts. They all knew the now near legendary story of a New York attorney using it to write a brief and how it made up cases. As several students stated, ChatGPT operated on the principle of “garbage in, garbage out.”
We ended the classes unimpressed for now with the ChatGPT revolution, but we did come up with ten rules that may be useful to teachers and students as they experiment with AI in the classroom:
- It’s useful to get a conglomeration or talk about ideas surrounding a topic; good for brainstorming; creates outlines for a paper. Students found ChatGPT useful for gathering information, or to start exploring a topic. If you are totally stumped about where or how to begin, it may help.
- It makes essay writing and assignments easier to begin with, but it’s not very helpful with the midst of the work. Once one uses ChatGPT to get started, it quickly loses its value. It does not replace the readings in class and it does not help you answer specific research questions that are specific to the class or subject matter.
- Use it as a starting point, rather than relying on it as a primary source; gets the ball rolling. None of the students said that they would rely on ChatGPT as a primary research or source tool. There are too many problems with its information or the information is limited and therefore outside research is needed.
- It can be used as a way to get an opposing point of view. Some students thought that the use of ChatGPT to generate pro and con bullet points on issues helped them think about opposing viewpoints or perspectives. However, they did not feel or believe that these pro and con points provided much information regarding who and why individuals hold these views.
- It’s very helpful when drafting; use it to check grammar or spell checks or use it to summarize readings. Many found ChatGPT to serve as a proofreader or spellchecker. In addition, some thought it could be used in terms of summarizing the main points for some class readings, but again they did not see it as a substitute. A few said they used it before or after reading an assignment to help them look for or review key points.
- If you use it for an assignment, you have to disclose and cite it; fact check everything. My students treated ChatGPT both as a search engine and a source of knowledge. They felt it important to explain or declare the search query as a way of explaining the results they received, and they agreed that it needs to be cited as a source, but it could not stand alone as a source of information unless corroborated.
- Limited in terms of sources (none), superficial, and does not address normative issues well. Students noted how limited its use as a way to bibliographical research and found mostly the results were very thin in terms of substantive knowledge. They also thought that it fared poorly in terms of helping them address normative questions or provide information for argumentation and persuasion assignments.
- Better if you role play. A few students thought ChatGPT worked better if you asked it, for example, to do a role play of two individuals debating a topic, such as whether capitalism is ethical.
- Use it as a review after/before class for summaries. ChatGPT performed well to help students fill in class notes to check to see if they missed any major points or issues on the topic. It helped with class outlines.
- Biased answers. Repeatedly, students criticized the biases in the information they received. From my perspective, this helped them become more critical thinkers and consumers of information, but for them it reinforced their concerns that ChatGPT had limited value on many topics.
- Does not in itself promote critical thinking; does not provide new insights — it relies on status quo knowledge. Students were emphatic that the use of ChatGPT alone does not produce or encourage critical thinking. Its superficial answers actually dumbed down many points. They also did not see how its use produced “new knowledge” since it was simply a conglomeration of often dated or biased information.
David Schultz is a distinguished professor at Hamline University. He teaches in political science, legal studies and environmental studies.
