For the past few months, I have been working heavily with ChatGPT (the paid version). It’s open all day long as I work on various things, and I repeatedly turn to it for various tasks and queries.
Up until recently, that is how I have seen and used LLMs, as a tool that I make use of as I do my work.
However, over the past few weeks, I’ve come to realize that ChatGPT is no longer a tool, because instead of something that can help me do my work, it is now demonstrating to me that it can do a lot of my work better than I can.
It can translate better than me.
It can read better than me.
It can write better than me.
And in many ways, it can think better than me.
Long gone are the days of hallucinated responses and awkward prose. Now instead of me noticing the faults in ChatGPT’s responses, it is ChatGPT that is pointing out the flaws in my ideas, and my translations, and my writing.
These are core skills that historians are supposed to be good at. . .
So, let me be (perhaps) the first historian to admit it. ChatGPT is now better at those skills than I am.
Having reached this stage of enlightenment, I’m going to give my poor brain and wounded ego a break for a bit and probably won’t post for a couple of months, but I’ll be back. . .
Strange times…
I still don’t get satisfactory answers from LLMs about my own topics of research and am surprised they can do much with Vietnamese just because the majority of Vietnamese written materials have never been digitised. Maybe it’s because I have never paid for an LLM so I don’t know how good the paid versions are, but I think maybe it’s also because I dig into extremely obscure topics that have passed right under the digital nose of an LLM. Even if an LLM has a monopoly over received knowledge and can outthink me as well as it does you, I still consider it to be missing huge areas of human knowledge that it does not have access to – either because they exist only in oral form or because they are written or spoken in a language that it has not been trained to recognise – for instance the traditional knowledge of the Thai ritual specialists in Thanh Hóa. These people have a lifetime of epic poetry and traditional knowledge committed to memory, and I don’t see any LLM being able to comment on it in more than very general terms acknowledging its existence.
Anyway, meditation is a good habit – one hour a day is my goal although at the moment I am stuck at about 40 minutes.
Think of it in terms of “AI creep.”
A few months ago, if I asked ChatGPT for sources, it would say, check Gallica. Now it goes in and gets them for me.
A few months ago, ChatGPT struggled to give me consistent editing information, now you can just throw an entire paper at it and say “make everything APA 7 consistent.”
Similarly, when I ask for advice about how the paper could be improved, after it gives me that advice (and this is where I can see that it is smarter than me because it points out things that I didn’t see/think of), I can just say “ok, go make those changes.”
The above issues alone dramatically change how we do what we do. Among other things, it can significantly speed up the time of producing a paper. And I think it will continue to “creep” in that direction.
What I left out was the actual writing. However, if you have the materials you are working with in digital form (and even hand-scribbled notes) you can then feed that material to it and give it directions.
Yes, transcribing Black Tai script is probably beyond it’s capability, although it’s totally possible to develop that technology, but beyond that. . .
And to say more about it being “smarter.” Ok, it can’t figure out the problems with 100+ years of scholarship on Sanfoqi/Srivijaya (because it’s knowledge is based on what’s out there), but a century of scholars haven’t been able to figure that out either (even though it’s not actually all that difficult).
So, it’s not as intelligent as people who work on and figure out obscure topics, but that is just a tiny percentage of the scholars/scholarship out there. Meanwhile, there are tons and tons of historical writings that don’t do that.
Further, once you have something down in print, it can take over and do much better than us. So, all of the posts that I just wrote on the historiography of Srivijaya — an LLM can now do much better than me in organizing and presenting that information.
I’ll ask the obvious: have you uploaded your posts on the historiography of Srivijaya to an LLM and asked it to consolidate them into a single manuscript suitable for submission to a journal? If so, what’s your opinion of the results?
I haven’t tried doing it with those posts, but that’s something it can do well now. It can determine if your argument is logical, or if there is a better way to make it. I also find it helpful when it points out that the way I say things could be offensive to some. . . 🙂
From what I’ve done so far, I don’t like asking it to write directly from the primary sources.
However, going forward, there is probably not going to be any way to differentiate between someone who feeds an LLM some sources and prompts it to write a paper based on them and someone who writes out what they want to say about the sources and then asks the LLM to clean/fix up what they have written (and to consult the sources in doing so to verify the claims and to see if there is more that the author has missed).
I would like to think that the latter approach would leave some space for human perception and ingenuity, but I’m not sure if that is the case.
Ultimately, the space for human ingenuity might come in the way we “mix” what we feed an LLM before prompting it. I still think it’s going to be the case that the more people know, the more they will be able to innovate with an LLM, while the less one knows, the more generic one’s output will be.