In the past week, I’ve had a few “Wow!” moments related to LLMs, AI, and digital knowledge, and so I thought I’d document that here. The comments I will make refer to the paid version of ChatGPT (now with Thinking 4.2) and the free version of Grok, and I also use Gemini for transcribing Chinese text, which it generally is excellent at.
First a mea culpa. I recently complained about ChatGPT and said that I was considering cancelling my subscription, however, dumbass that I am, I think I had it on “auto” mode rather than “thinking”. . . And for the things I was trying to do, that didn’t work, but instead, just led to me end up arguing with it.
That said, I still find Grok’s thinking to be better than ChatGPT’s, but the latter can still do a lot.
Grok is “That Kind” of College Student
On the topic the thinking of LLMs, if you watch an LLM “think,” you can see that it is now “doing” things rather than simply “checking” things. It opens pdfs, it zooms in to get a higher-resolution screen capture, it examines what is in the images it captured, etc. and if it can’t do certain things (like access a website or a complete a download), it tries various alternative techniques, etc.
In other words, we can see the “agents” now at work.
This is one area where I find Grok to be superior, as it is like “that kind” of college student. Some of you undoubtedly know what I mean. For those who don’t, let me explain.
I grew up near a small college town, and at that time, the drinking age was 18. The college was on a hill up from the town, and in between the college and the town was a small convenience store which the college students used to walk down to on Friday and Saturday nights to buy beer and snacks to take back to their dorm rooms.
So, what underage high school students would do, was to stand near the convenience store and wait for “that kind” of college student to walk by, you know, the one who would agree to buy an underage kid a six-pack of beer, and who would also pocket the change. . .
Grok is “that kind” of college student. So, if there is something that ChatGPT tells you it can’t do, go stand outside the convenience store and wait for Grok to walk by. It’s not a guarantee, but your chances of getting that six-pack are still better with Grok.
Ctext goes AI
Today I went to check something on Ctext, an online site of digitized Chinese historical sources, and found that a page of Chinese text that I looked at just yesterday, now contains accompanying translated text in English, courtesy of AI.
ChatGPT Blows My Mind
The other day, I was working on an early-nineteenth-century Vietnamese text in classical Chinese that also contains Nôm characters (that is, characters in the Vietnamese demotic script).
I used Gemini to scan the passage I was working on from a poor-quality scan that I have. It couldn’t understand the Nôm characters, so I then looked them up in an online Nôm lookup tool.
I then threw a passage into ChatGPT and asked it to translate it, just to see what it could do.
This text has never been digitized, so when I put the text into Google and searched, nothing came up. Further, I did not give ChatGPT any context or even tell it what the title of the work was.
That said, there is a Vietnamese translation of this text in a special issue of a journal that is available online in the form of pdf files. A scan of the text is also included there, but it is of very poor quality. I don’t think even the best OCR tools could decipher it.
So, I threw the passage in and then sat back and watched ChatGPT “think.” It quickly identified the text as similar to Qing dynasty-era navigational texts and tried to find information in various Chinese websites, then, to my total surprise, it figured out that the text must be the same text as the one in the Vietnamese journal, however, it was unable to open the pdf there for some reason.
That’s f’ing mind blowing!!!
Further, yes it got some of the Nôm characters wrong, however, it correctly figured out some of the places that they referred to.
Zdic goes Nôm
On that note of Nôm characters, I was also consulting Zdic, the best online Chinese-Chinese dictionary (and it also has brief English definitions). And I looked up a character in this text that I saw from the Vietnamese translation of the document I was working on was supposed to mean “mouse,” but I was unfamiliar with the character (chu 𤝞). I wondered if it was perhaps an archaic Chinese term for mouse.
When I looked it up on the Chinese-Chinese dictionary, Zdic, again, much to my surprise, it indicated that this was the Vietnamese character for “chuột,” meaning “mouse.”
Maybe I have not been looking in the right places, but in using Zdic for close to 20 years, I had never seen Nôm characters documented on the Zdic site.
The Coming of “Universal Knowledge”
These “Wow!” moments all point to a coming development, something that we can think of as “universal knowledge.”
Everything that is “out there” is becoming connected, and it is all translatable into English.
In trying to decipher a relatively obscure undigitized Vietnamese text in classical Chinese that contains Nôm characters, I still have to make certain connections myself, but that’s clearly not going to be necessary for much longer.
The agents running in the background as ChatGPT “thinks” are already chipping away at this, and with Nôm on Zdic (where LLMs also get a lot of their English equivalents for Chinese terms), they are getting closer and closer.
Viral Essays
There are an essay and a follow-up essay that went viral this past week: “Academics Need to Wake up on AI” and “Academics Need to Wake up on AI, Part II.” The author, Alexander Kustov, states that:
“I study immigration and public opinion, not AI. But I’ve spent the last few months watching AI transform my own research workflow, and I have some things to say to my colleagues. For the first time in my life, I genuinely do not know what academia will look like in five years. Even if progress stalls completely and we are stuck with the current models forever, the changes already in motion will transform my field of academic research and publishing beyond recognition. The status quo is unsustainable. It may take time, because academia is the most dispositionally conservative institution on the planet. But it will change.”
He then goes on to discuss this in more detail.
Personally, I find pretty much everything that he said to be common sense, however it caused quite a storm, particularly in the space of the hell-hole known as Bluesky.
A few months ago, I wrote about how I think the History profession as we know it is “toast” because of what LLMs can already do, and as I see it, this is what Kustov is saying, from different perspectives, about the Social Sciences.
Vietnamese History through Video Games
It was announced this week that “Vietnam wants 5,000 developers to turn its history into video games.”
Good idea.
And an LLM Tip
When you use an LLM, creating a good prompt is important. What I now do is to explain what I want to do, and then ask the LLM to create the best prompt possible for that. Finally, I then run the prompt that it generates.
For basic queries, that’s not necessary, but if there is something that you really want to dig into, then I find that this technique works well.
This might be common sense to some, but I’m sure that there are others who are unaware of this.
Thanks for writing this; you’ve reminded me that I need to finish a piece on “the death of a discipline” (political science). I see AI as the final nail in the coffin for a field that has become increasingly irrelevant because of academia’s obsolete incentive structure and conservative gatekeeping.
I hadn’t thought about political science, but thinking about it now, it seems to me like its output would be one of the most easily replaceable by AI.
While there are different ways of doing it, in general, one doesn’t have to obtain non-digitized information in archives or from interviews or fieldwork. Instead, one can use publicly available information and then interpret it through a theoretical framework.
That’s far too easy for an LLM.
The defense that historians make against AI is that “We base our scholarship on hand-written letters in indecipherable gibberish that are in a locked box under a bed in a secret cabin on a hidden island in Tierra del Fuego.” What is the defense that political scientists make?
I’m not aware of political scientists making a defense. Maybe some are sounding the alarm somewhere, but not in venues I pay attention to.
My dissertation combined field interviews with a statistical analysis of survey data. I can see AI doing the following:
I assume that within two years we’ll see AI producing political science dissertations from start to finish, with “authors” doing nothing but submitting prompts — I see many dissertations that are nothing but descriptive historical summaries based on secondary source material, a task that AI can already perform relatively well. Maybe grad students are doing this already. I know it’s currently happening in a variety of fields with exploding rates of journal article submissions, which is why I’m eagerly anticipating the AI-induced collapse of academia’s publish-or-perish incentive structure.