I see a lot of academics brushing off AI, thinking that it’s not a threat. How do they reach that conclusion?
As far as I can tell, in many cases they go on free versions of AI platforms, like ChatGPT, try a few things out, get back some hallucinations, and then conclude that there is nothing to worry about.
That, however, is itself a kind of hallucination. Let me explain why.
To take ChatGPT as an example, it has different tiers. There is a free version, a plus version ($20 a month), and a pro version ($200 a month).
The free version of ChatGPT now offers access to the GPT-5 model, but with usage limits, after which it switches to a less capable, faster model called GPT-5 mini.
The plus version offers access to GPT 5 Thinking (thinks longer for better answers).
The pro version offers access to Deep Research, OpenAI’s advanced autonomous research tool that performs in-depth, multi-source analysis and generates detailed reports with citations, unlike standard web searches.
These distinctions are always changing, and GPT 5 Thinking now has access to some of the deep research capabilities that were previously only available at the pro tier, etc.
In any case, my point is that you cannot make an informed judgement about AI (or LLMs) if you only experiment with the free versions.
There is an historian in Canada by the name of Mark Humphries who has been working with AI for years and trying to figure out how it can contribute to historical scholarship. And he has a Substack called Generative History.
In February of this year, he wrote a Substack post entitled “Is this the Last Generation of Historians?”
I would encourage every historian to read that post, as it is very informative.
In that post, Humphries links to a query that he made using Deep Research in the pro version of ChatGPT. This was his query:
Write a historiographical essay comparing and contrasting Canadian and American approaches to the history of fur trade families in North America. Be sure to only use peer-reviewed academic secondary sources and be sure to refer to key historians by name. Aim to be comprehensive.
Deep Research then asked him for some clarification (the free and plus versions do not do this), which he provided, and then it produced the essay for him.
Humphries then stated that:
Its response is very good. But consider that right now it can only access full-text sources that aren’t paywalled, which is why the bibliography is so limited. Even so, it included lots of scholars it could not actually read…and did a good job with their works. Think forward to what this will look like once OpenAI actually allows users to upload files and Deep Research can access journal repositories and e-library resources on its own.
There is more to this article that discusses the current capabilities of Deep Research. However, I thought that it would be valuable to compare the historiographical essay that Deep Research produced for Humphries with the responses of the free and plus tiers to the same query.
So, I put the same query into the free and plus versions, and have linked those below together with Humphries’ version created with Deep Research.
I think you can see how they differ. The free version gives you an essay with limited mention of sources (and I didn’t check to see if they are legit or hallucinated).
The plus version is sort of like a “draft file” that we might have created the old school way by writing down our notes in outline form and including the citations (or links to them in this case) before we then sat down and “connected the dots” by writing a narrative and putting in the footnotes (but you can then, of course, ask the plus version to create a narrative).
Meanwhile, the Deep Research version pretty much does it all. It has a narrative, links to the citations, and provides a bibliography. And as Humphries wrote, as these AI companies work out deals with publishers, all of this will become more sophisticated as Deep Research will have access to journal databases and academic press holdings.
As I have been arguing, I think the “draft file” that the plus version currently creates is already sufficient to enable students to rely on AI to produce all the writings they need to get a BA degree (it’s even possible with the free version, but it takes a bit more effort).
With Deep Research. . . I see MA theses and PhD dissertations easily getting written.
Ah, but you say, graduate students are not going to be able to pay $200 a month!! So, we’re ok!!
They are most likely not going to need to, as over time, as is already happening, the more advanced capabilities will trickle down to the lower tiers.
Further, some of the AI research products that Andy Stapleton introduces, a topic I addressed in an earlier post, are also in the process of establishing arrangements with databases like Google Scholar, so in the end, there will probably still be different tiers, but the affordable ones will be sufficient to get the job done for many.
Alternately, I can see some grad students chipping in together to share an expensive account.
So, to return to Humphries’ question, “Is this the Last Generation of Historians?”
In their current form, yes. This is easy to see. All you have to do is to look beyond the free tier.
The “pro” version cites Wikipedia
Thank you for the comment.
It doesn’t cite Wikipedia. It cites Richard White’s seminal work, “The Middle Ground: Indians, Empires, and Republics in the Great Lakes Region, 1650–1815,” and then it contains a link to the Wikipedia page where it found this work mentioned/discussed.
As Humphries stated, and as I cited him in this post:
“Its response is very good. But consider that right now it can only access full-text sources that aren’t paywalled, which is why the bibliography is so limited. Even so, it included lots of scholars it could not actually read…and did a good job with their works. Think forward to what this will look like once OpenAI actually allows users to upload files and Deep Research can access journal repositories and e-library resources on its own.”
Richard White’s work is a published book. The full text is not available online. So, the LLM can’t “read” it, but as Humphries stated, it still did a good job in figuring out about it by seeing what Wikipedia said about it (among perhaps other online mentions).
Currently, academic publishers around the world are working on agreements with AI companies that will open up their repositories of full-text sources that are behind paywalls. So, if ChatGPT Pro can already do well without actually having access to a source, Humphries’ point is that we can expect it to do even better when it has direct access to those sources, and that is coming sooner rather than later.
Finally, how many times have scholars found a source on Wikipedia that they then cited in their work? Is it 30,887,296 times now?
For all of the above reasons, the fact that ChatGPT becomes aware of a key source for a topic because it is mentioned and discussed on Wikipedia in no way diminishes from this extremely powerful technology, to think otherwise would be a fatal miscalculation.
And on that topic of fatal miscalculations, when Wikipedia first came out in 2001, academics ridiculed and dismissed it. However, by 2005/6, when it started to regularly appear at the top of search results, its central importance for the future of knowledge became obvious.
Rather than continuing to dismiss it, I think universities should have started a “Digital Knowledge Manhattan Project” to make Wikipedia as solid and thorough as possible (it could still be done, btw), because it was obvious that the future is digital and open, not print and paywalled.
Now we have AI. . . What do we do? Repeat our successful Wikipedia approach? That’s what I see many people doing in my field, and is why I am writing these posts to say “Wake up!!” Because this time the stakes are truly existential.
This is still very unconvincing. Even the most expensive paid tier, which we had access to at my former institution, did not have access to archival documents, the “primary documents” that it did have access to had already been selected and parsed by human historians, and thus given a rank and priority. Moreover, even the paid version, while it can synthesise different interpretations in the scholarship, it cannot make reasoned judgements among that scholarship. The work of Albert Fishlow and Richard White on railways are very different, one is reliant on the other, and their conclusions are incongruous. I do not count as serious professional history works that are simply syntheses, human or AI, of already existing scholarship or published primary sources. This is a tertiary function of a historian, at best.
Now, will indolent, poorly informed, cost-cutting deans and administrators want to replace professional historians with AI that can produce a poor facsimile of what actual modern historians do – go to archives, sort, select, edit, collate, translate, parse and synthesise documents – yes they will probably want to do that. But they will be getting precisely that – a poor facsimile. In most modern fields tiny fractions of the available archival materials have been made available. In fields with a closed corpus – medieval English history, or fields with vast digitisation resources – like Early English Books online and Eighteenth Century history online (with tens of millions of digitised pages) the threat might be greater. But hundreds of millions of pages of the British National Archive and India Office Library remain undigitised and won’t be any time soon even with financial investments an order of magnitude greater than they are now. Are the same barbarian deans and administrators likely to want to replace historians qua *history teachers* with apps that can give customised content and adaptive tests in the style of Duolingo and other third-rate applications> Probably. But these are both very different to saying that AI is a threat to professional historians qua history researchers or to historians qua history teachers.
AI in the field of modern philosophy is beyond useless. There is no AI model even remotely close to being able to do the work that Kit Fine or Hartry Field or Tim Maudlin do; or Saul Kripke or David Lewis or Wilfrid Sellars or Willard Van Orman Quine or Philippa Foot or Iris Murdorch or Wilfrid Sellars did. Even the most powerful modern LLMs and predictive AI models are incapable of the kind of creative analytical reasoning these scholars and people are famous for. AI poses various philosophical problems. But it is not yet a problem for philosophy as such, which it is completely incapable of.
A further question is even if all of this is true – administrators eliminate historians as researchers (because they falsely believe that AI can do historical research) and they eliminate history teachers (because they mistakenly believe apps can do a better job) – will it matter very much? Have historians done very much lately to arrest the descent of our shared culture into cruelty, stupidity, and barbarism even a little bit? I have seen no evidence of it. Indeed, many of my former coworkers were little more than well credentialed philistines and defenders of violent thuggery and genocide. The beginning of the moral and intellectual collapse of university education long predates AI. If AI hastens that collapse (which it may, mainly because of the stupidity and cupidity of administrators, not the power of AI to do history research or to teach history) then historians can do something useful like market gardening, share cropping, animal husbandry, or whatever.
There is a reporter by the name of Karen Hao who has written a good book on the “dark side” of AI and the Silicon Valley companies behind it, called “Empire of AI.” She has also been giving lots of interviews, all of which are good, so if you come across one, check it out.
However, I was listening to one recently where she said this:
44:14
“And so there will be certain things that will certainly, like AI models, will be technically competent at replacing a human. There will be many other things that it will not be. But that won’t necessarily have a bearing on whether that person keeps their job anyway because ultimately it’s not actually AI taking your job. It’s humans. It’s an executive deciding that your job is now redundant.”
(https://www.youtube.com/watch?v=e70RT6c01M8&t=1949s)
That is EXACTLY the point I have been trying to make, and the reason why I am so alarmed is because I see a perfect storm of elements that have combined around universities that makes it extremely easy for a university administrator to make that decision about say History profs or even Departments (it was already happening before AI). The storm is slightly different in different parts of the world, but it’s all over the globe, and it’s what I’ve tried to highlight in recent posts.
Some of those elements are the failings of profs themselves, some of them are problems of how universities are run, some are demographic changes, another is the social change brought on by the Internet and social media, there is the accompanying declining interest in reading and writing in favor of visual/oral forms of communication, there are declining numbers of majors, there is online teaching, another is progressive pedagogical ideas, you have the rankings craze at some universities, etc.
All of these elements already make it relatively easy for administrators to determine that Historians are redundant, but it’s still been hard to get rid of them without an alternative. . . And now AI arrives on the scene. It’s shiny. It’s futuristic. Woke pedagogy can get behind it. Corporations say they need people who understand it. . . win win win win!!!
So, I keep seeing/hearing people saying “Oh, come on! AI can’t do what I can do!”
It doesn’t have to!! The only thing that needs to happen is for an administrator to think that it can. And 1) in this perfect storm, it’s pretty easy for them to come to that conclusion, and 2) administrators around the world are already ok’ing the use of AI “tutors” and other things like this:
“Working with Social and Decision Sciences Professor Daniel Oppenheimer on Socratic textbooks. These AI-powered books deliver personalized content and tutoring through conversational interactions while adapting to each student’s pace and interests using the Socratic method. This new approach provides instructors with detailed insights on individual and class-wide learning progress, enabling targeted interventions and curriculum adjustments.” (https://www.cmu.edu/dietrich/news/news-stories/2024/november/ai-humanities-social-sciences.html?utm_source=chatgpt.com)
Is anyone going to be able to convince an administrator in a few years that a prof (who you have to pay) who offers online courses that consist of videos that s/he filmed several years ago and just keeps recycling, and who rarely or never responds to student emails, is better than a personalized Socratic textbook (which you can make $$ by selling)??
Back where I used to work, the community colleges have turned heavily to online. Are people really going to be able to make a convincing case that this is superior to a Socratic app that not only develops critical thinking (“based on proven pedagogical research conducted by [Elite University] scientists”) but also assesses with complete neutrality, to say nothing about all of the progressive logic about things like accessibility that is already used to legitimize online teaching, and the financial benefits?
There is a young guy who writes for the Atlantic, Tyler Austin Harper, who recently wrote a piece where he argued that universities also should go full no-tech. No wifi, no laptops, of course no AI, etc. (I don’t have a subscription, but he shares his main ideas on X).
I don’t possibly see that working on a large scale, however, I could imagine a small college getting enough people to agree to that, and I think the creation of “parallel institutions” is one possible solution/alternative.
[Update: Just saw this report on how Campion College, a small Roman Catholic liberal arts college in Australia is taking a stand on AI usage: https://www.afr.com/work-and-careers/education/how-a-liberal-art-college-is-fighting-the-ai-cheating-crisis-20250910-p5mu1q. This is certainly one approach, and I can see it working in this type of university. In fact, providing an alternative to AI could theoretically be a life saver for some of the small Christian colleges out there that are currently struggling.]
At the end of that interview, Karen Hao says collective action is the best solution. People in a group have to make demands that things be one way or another.
That makes sense, but it can only happen when people actually acknowledge and talk about the problem. My frustration comes from seeing many people dismiss this issue. People are just responding from their egos – “Ai is not better than ME!!!”
Here I’m probably influenced by witnessing the decline of History majors in the 2010s. My colleagues in my own department couldn’t wrap their heads around that even though I showed them the data semester after semester. That being the case, there was not surprisingly no discussion between departments or at the university level, and of course no discussions between universities. Years went by like that until finally I was in a meeting of university major advisors and an admin person showed us the data for major number across the various departments: like History, they had been declining in numerous departments for years, yet I had never heard anyone anywhere say a single word about this. Then it was shortly after that point that Ben Schmidt published his article on data that showed the decline in majors, and then finally, “suddenly” people became aware of the problem. It’s like academics can’t understand anything unless it is communicated to them in a document written with citations. However, that was like 7 years into the problem already. And the response was to scream “But we teach critical thinking,” and nothing changed. . .
So, I think fields like History are a long way from collective action (whatever that might be) and are probably incapable of it, or at least not capable of doing something in time to avert the worse. However, as a human being who can comprehend reality fairly well without needing to wait for a study with citations, I still want to figure out what to do. What do we do with research? What do we do with teaching? What do we do with assessment?
Re: D Insor
“AI in the field of modern philosophy is beyond useless. . . Even the most powerful modern LLMs and predictive AI models are incapable of the kind of creative analytical reasoning these scholars and people are famous for. AI poses various philosophical problems. But it is not yet a problem for philosophy as such, which it is completely incapable of.”
“Are the same barbarian deans and administrators likely to want to replace historians qua *history teachers* with apps that can give customised content and adaptive tests in the style of Duolingo and other third-rate applications> Probably. But these are both very different to saying that AI is a threat to professional historians qua history researchers or to historians qua history teachers.”
Have you seen what’s happened to the numbers of U.S. undergraduate history and philosophy majors in the last few decades? Fewer students taking history and philosophy courses equals U.S. universities employing fewer people with PhDs in those fields. So unless one is independently wealthy, getting a history or philosophy PhD is a very stupid financial decision.
Under the corporate university model (which is effectively universal across U.S. higher education), administrators don’t care about “serious,” “professional,” or “actual” historians, they are only interested in “good enough,” which means somehow helping generate net revenue.
This discussion has been conflating issues which are separate. 1) Can AI do historical research? (No, not really) 2) Can AI teach history (maybe, kind of, probably not very well (I personally doubt it, but it seems like an empirical questions) 3) Will administrators use AI as an alibi to replace historians? Yes, probably. (3) is where the real threat is. When I say I am not convinced above it is by AI as a threat to research and to responsible teaching. I don’t think it is a threat to either. But I absolutely agree with (3), which is a question with an answer almost entirely independent of how you answer (1) or (2). Are the barbarians at the gates? Certainly. Where I went to graduate school, history was the most popular undergraduate major and had been for years (late in my time there political science came to rival it). And where I taught for the last decade, the history major had been growing year on year for about five years as had overall history enrolment and TT faculty hiring. No generalisations are possible from a sample of one. But where I studied and taught, people did care about serious, professional, actual historians.